How aadi1011/traffic-light-optimization-with-reinforcement-learning works
Main Components
• Traffic Simulation Environment: Represented by the SUMO configuration and route files in the “intersection” directory (e.g., sumocfg, environment.net.xml, episode_routes.rou.xml). This component simulates the intersection and generates traffic flow data.
• Deep Q-Learning Agent: Central to the system, this agent (“Model”, “Memory”, and related files like memory.py, model.py, training_settings.ini) is responsible for decision-making. It has a state processing module (using discretized traffic states), an action selection mechanism (choosing traffic light phases), and a reward evaluation process (reducing cumulative waiting time).
• Simulation Controllers: The training and testing loops are handled by distinct entry point files (training_main.py, testing_main.py, training_simulation.py, and testing_simulation.py). They orchestrate the simulation cycles, interacting with SUMO and passing simulation data to the agent.
• Traffic Generator: Indicated by generator.py, this component deals with vehicle and route definitions for each simulation episode.
• Visualization & Utilities: Using visualization.py and utils.py, these components support plotting performance data and provide file handling or other helper functions.
• Additional Modules: gpucheck.py ensures the proper GPU environment, and there are multiple model directories under “models” that may represent different experimental configurations or trained model versions.
Component Relationships and Data Flow
• Experience Replay Memory which stores state-action-reward transitions to be used for training the neural network.
• A separate “training loop” section where the model is updated based on batches from memory. Show that the “training_main.py” script, along with “training_simulation.py”, is responsible for driving this cycle.
• That “testing_main.py” and “testing_simulation.py” are used for evaluation purposes.
• Side components like the Traffic Generator that feeds the intersection simulation with vehicle routes, and the Visualization module which plots reward curves, queue lengths, or delay data as logged by the tests (as evidenced by the “plot_*” files in the models folders).
Architectural Patterns and Design Principles
• Separation of concerns: Clearly distinguish between simulation control, learning logic, utility and visualization modules.
• Real-time Interaction with External Services: Include SUMO as an external simulation engine interacting with the internal training/testing loops.