Adaptive Safety Evaluation for Connected and Automated Vehicles with Sparse Control Variates

Introduction

This project contains the source code and data for the paper titled "Adaptive Safety Evaluation for Connected and Automated Vehicles with Sparse Control Variates".

Code Structure

Note that data/ and results/ are publicly available on the Hugging Face Hub.

|- data/
|- figures/
|- results/
    |- example_results/
|- utils/
    |- __init__.py
    |- config.py
    |- evaluation.py
    |- importance_function.py
    |- index.py
    |- maneuver_challenge.py
    |- state.py
    |- surrogate_model.py
    |- test.py
bootstrap_NADE.py
bootstrap_NDE.py
bootstrap_SCV.py
Makefile
README.md
requirements.txt
results_analysis.ipynb
test_NADE_SCV.py
test_NDE.py

Installation

Please create a new environment via conda and install the required packages as follows. Note that this project can be used on both Windows and Linux.

conda create -n scv python=3.10
conda activate scv
pip install -r requirements.txt

Usage

Please first configure the settings in Makefile (number of CPU cores, number of tests, etc.), and then use the following commands to run experiments.

make all

Instead, you can run testing and evaluation separately as follows.

make test # test AV in NDE and NADE
make eval # bootstrap testing results of NDE, NADE and SCV

For development, to quickly check the pipeline, please use make dev.

Results Analysis

Please run results_analysis.ipynb for results analysis. The example results are provided in results/example_results/, and the example figures are shown below.

• NDE vs. NADE

• NADE vs. SCV

• Bootstrapped required number of tests

Contributing

Any contributions are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Citation

Yang, J., Sun, H., He, H., Zhang, Y., Liu, H. X., & Feng, S. (2023). Adaptive safety evaluation for connected and automated vehicles with sparse control variates. IEEE Transactions on Intelligent Transportation Systems, 25(2), 1761-1773. https://www.doi.org/10.1109/TITS.2023.3317078

@article{yang2023adaptive,
  title={Adaptive safety evaluation for connected and automated vehicles with sparse control variates},
  author={Yang, Jingxuan and Sun, Haowei and He, Honglin and Zhang, Yi and Liu, Henry X and Feng, Shuo},
  journal={IEEE Transactions on Intelligent Transportation Systems},
  volume={25},
  number={2},
  pages={1761--1773},
  year={2023},
  publisher={IEEE}
}

License

This code is licensed under the PolyForm Noncommercial License 1.0.0.

Developers

Jingxuan Yang ([email protected])

For help or issues using the code, please create an issue for this repository or contact Jingxuan Yang ([email protected]).

Contact

For general questions about the paper, please contact Shuo Feng ([email protected]).