A Provable Safety Metric for Autonomous Driving via Deep Reinforcement Learning and PAC Verification

Introduction of the Project

About

This project contains the source code and data for the paper titled "A Provable Safety Metric for Autonomous Driving via Deep Reinforcement Learning and PAC Verification". We propose a real-time automonous driving safety metric called Provable Minimum Collision Time (PMCT) that provably predict the minimum time to collision from the inputed initial state.

Code Structure

PMCT-master/
|__ conf: experiment configurations
|__ maps: maps for SUMO simulator
|__ source_data: source data for constructing NDE testing 
|__ mtlsp: simulation platform
|__ envs: testing environments
|__ controller: vehicle controllers (e.g. IDM)
|__ utility_functions: utility functions
|__ safety_metric_tool: tools of calculating safety metrics results
|__ network_structure: definition of the DRL-based network
|__ param_save: the network parameters
|__ result: results of all the experiments 
|__ training_and_verification_main.py: main function for training and verifying the network
|__ provability_experiment_main.py: main function for experiments on PMCT provability validation 
|__ safety_metric_experiment_main.py: main function for experiments on comparison between PMCT and other safety metrics
|__ collision_risk_experiment_main.py: main function for calculating the system collision risk upper bound
|__ occurance_probability_calculation.py: function for calculating occurance probability of initial states by subsets
|__ generate_failure_data: function for generation of failure data used in network verification as a supplement
|__ nadeinfoextractor.py: information extractor for logging experiment information
|__ requirements.txt: required packages

Installation

Pre-requirements

• Python installation
  • This repository is developed and tested under python 3.10.16 on Ubuntu 20.04.6 system.

Installation and configuration

Clone this repository

git clone https://github.com/fanlx2004/PMCT-master-tmp

Create a new virtual environment (Optional)

To ensure high flexibility, it is recommended to use a virtual environment when running this repository. To set up the virtual environment, please follow the commands provided below:

virtualenv venv
source venv/bin/activate

Install all required packages

Due to the compatibility issue betwen gym and python setuptools, user should run follow commands to install a specific version of setuptools first:

pip install setuptools==65.5.0

To install the Python packages required for this repository, execute the command provided below:

pip install -r requirements.txt

Usage

The project includes code (network training and verification, experiments) and experiment outcomes (net paramaters, datas and figures). The project constructs the Q-network by training and verifying, calculates the system collision risk upper bound under NDE environment, and conducts experiments of PMCT probability and metrics comparison. Several main functions are provided for the above usages.

Usage 1 (Network training and verification)

We utilize a DRL-based network (called Q-network) to solve a provable state set partition problem. The structure of the Q-network and its training agent are defined in ./network_structure/Q_net.py. We have a version of network paramaters that have already been trained and verified successfully, saved as ./param_save/net_params_final.pth. To obtain a new version of network paramaters, run the training and verification script with the following command:

python training_and_verification_main.py

The network paramaters under training is saved as ./param_save/net_params_undertraining.pth.

In addition, some pre-collected data from each subsets that the network fails to predict their minimum collision step previously is used in verification as a supplement in case the verification process can't end. These data can be generated by running the following command:

python generate_failure_data.py

However, due to the high time consumption of the generation process, we suggest using the data we have already generated, saved in ./result/failure_data.

Usage 2 (PMCT experiments)

The function to calculate PMCT based on an inputed initial state is defined in ./safety_metric_tool/PMCT_solver.py. We provide main functions to repeat the experiments on PMCT shown in our paper.

Usage 2.1 (Experiments on the provability of PMCT)

To conduct the experiments on verifying the provability of PMCT through scenario simulations, run the following command:

python provability_experiment_main.py

If you want to re-generate the true collision times and predicted values used in the experiments, set the boolean varible generate_data to True in Line 16 of provability_experiment_main.py.

The experiment results is saved in ./result/provability_experiment_data, and the figures showing comparison of true and predicted values are saved in ./result/experiment_figure/provability_experiment.

Usage 2.2 (Experiments on safety metrics performance comparison)

We compare the performance in real-time safety assessment between the proposed PMCT and other two safety metrics PET and MPrISM through scenario simulations. The functions to calculate PET and MPrISM is in ./safety_metric_tool/PET_solver.py and ./safety_metric_tool/MPrISM_solver.py

Run the following command to repeat the experiment and draw the ROC and PR curves:

python safety_metric_experiment_main.py

The ROC and PR curves are saved in ./result/experiment_figure/ROC_PR.

If you want to re-generate the true collision times and predicted values used in the experiments, set the boolean varibles to True in Line 22-25 of safety_metric_experiment_main.py. However, due to the superior time consumption calculating MPrISM, we suggest using the data we saved in ./result/safety_metric_experiment_data.

Usage 3 (System collision risk upper bound calculation)

The automonous driving system collision risk in finite-time scale is estimated and proved an upper bound in the project.

The occurance probability of the initial states and its distribution by subsets are first derived based on naturalistic driving environment (NDE), saved in ./result/occurance_probability_data. If you want to re-generate these data, run the following command:

python occurance_probability_calculation.py

Then we calculate the transition metrix of subsets, the collision risk expectations of subsets, system collision risk estimation and upper bound by the following command:

python collision_risk_experiment_main.py

The results are saved in ./result/collision_risk_result.

This command also conducts validation of the derived system collision risk upper bound by statistically computed ground-truth collision risk. If you want to re-generate the transition metrix or re-generate the ground-truth collision risk data, set the boolean varibles to True in Line 21 or Line 23 of collision_risk_experiment_main.py, respectively.

Developers

Lingxiang Fan ([email protected])

Linxuan He ([email protected])

For help or issues using the code, please create an issue for this repository or contact Lingxiang Fan ([email protected]).

Contact

For general questions about the paper, please contact Shuo Feng ([email protected]).