Traffic Light Optimization With Reinforcement Learning

This project offers a framework for optimizing traffic flow at complex intersections using a Deep Q-Learning Reinforcement Learning agent. By intelligently selecting traffic light phases, the agent aims to maximize traffic efficiency.

Deep Q-Learning Agent

• Framework: Q-Learning with a deep neural network.
• Context: Traffic signal control at a single intersection.
• Environment: Features a 4-way intersection with 4 incoming and outgoing lanes per arm, each 750 meters long. Traffic lights are positioned such that each arm has dedicated lanes for specific movements.
• Traffic Generation: Each episode generates 1000 cars following a dynamic pattern, contributing to the complexity of the environment.
• Agent (Traffic Signal Control System - TLCS):
  • State: Discretized representation of oncoming lanes, identifying vehicle presence in cells.
  • Action: Selection of traffic light phases from predetermined options, each lasting 10 seconds.
  • Reward: Based on cumulative waiting time reduction, incentivizing efficient traffic management.
  • Learning Mechanism: Utilizes Q-learning equation and a deep neural network to update action values and learn state-action relationships.

Getting Started

Follow these steps to set up the project on your local machine:

1. Download and install Anaconda from the official website.
2. Download and install SUMO from the official website.

Ensure your system meets the software version requirements listed below:

• Python 3.11
• SUMO traffic simulator 1.2.0
• tensorflow 2.16.1
• tensorflow-gpu 2.5.0
• matplotlib 3.6.0
• numpy 1.19.5

Running the Algorithm

1. Clone or download the repository.
2. Navigate to the root folder in the terminal.
3. Run python training_main.py to start the training process.

Code Structure

The project is structured around several classes handling different aspects of the training process:

• Model: Defines the deep neural network architecture.
• Memory: Manages experience replay for training stability.
• Simulation: Controls the simulation environment and interaction with SUMO.
• TrafficGenerator: Handles route definition for vehicles in each episode.
• Visualization: Provides functions for plotting data.
• Utils: Contains utility functions for file handling and directory management.

Settings Explained

The settings used during training and testing are specified in the training_settings.ini and testing_settings.ini files. Key parameters include episode duration, traffic generation details, neural network architecture, and simulation settings.

Contributors

This project was developed by:

• Aadith Sukumar
• Aniket Singh
• Apoorv Gupta

For additional information or inquiries, please feel free to open an issue on the repository's issues page.

Troubleshooting

macOS (Apple Silicon / M1, M2, M3)

sumo-gui shows a blank grey window (GLXBadContext errors)

The official SUMO .pkg installer uses an X11/FOX toolkit build that has broken OpenGL on Apple Silicon, even with XQuartz installed. Use the native arm64 binary that ships with the eclipse-sumo Python package instead:

pip install eclipse-sumo
export SUMO_HOME=$(python -c "import sumo; print(sumo.SUMO_HOME)")
export PATH=$SUMO_HOME/bin:$PATH

Add these exports to ~/.zshrc to make them permanent. Then run training or testing as normal.

ValueError: Argument(s) not recognized: {'lr': 0.001}

The lr keyword was removed in newer Keras versions. If you encounter this, update model.py line where the optimizer is compiled — replace Adam(lr=...) with Adam(learning_rate=...).

TypeError: MeanSquaredError.get_config() missing 1 required positional argument: 'self'

Pass the loss as an instance, not a class. In model.py, change loss=losses.MeanSquaredError to loss=losses.MeanSquaredError().

FileNotFoundError: No such file or directory: 'sumo-gui'

SUMO is not on your PATH or SUMO_HOME is not set. Set both environment variables as shown above before running.

References

• Amazon AWS Documentation
• Dr. Rahee Walambe - Symbiosis Institute of Technology, Symbiosis International (Deemed) University
• Dr. Smita Mahajan - Symbiosis Institute of Technology, Symbiosis International (Deemed) University
• Analytics Vidhya
• Towards Data Science (Medium)
• Andrea Vidali - University of Milano-Bicocca
• "Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems" - Xiao-Yang Liu, Ming Zhu, Sem Borst, Anwar Walid