How eeeeeclectic/traffic_signal_control works
Project Type and Purpose
• This is an open-source, Python-based framework and experiment suite for traffic signal control via multi-agent, graph-based reinforcement learning.
• It builds on SUMO (via TraCI) to simulate traffic, adds a modular “sumo_rl” library (environment, agents, graph-models), and provides “experiments/” scripts to drive training, evaluation, and plotting.
Main Architectural Components
A. External Simulation Service
• SUMO Simulator & TraCI API
B. Core Library (“sumo_rl/”)
1. environment/
– SumoEnvironment (Gymnasium/PettingZoo wrapper)
– TrafficSignal controller (phase, reward, observation)
– ObservationFunction abstractions
2. agents/
– QLearningAgent, PG_SingleAgent, PG_MultiAgent, DCRNN-enabled agents
3. models/
– base_model, dcrnn_cell, dcrnn_model, transformer_model
– util functions for graph construction, k-hop neighbors, temporal features
4. nets/
– Predefined network configs (.net.xml, .rou.xml, .sumocfg) for 1×1 up to real city grids
5. util/
– gen_route.py for generating SUMO route files
C. Experiment Orchestrator (“experiments/”)
– Python scripts (centralised_control.py, decentralised_control.py, DQN, PPO, Q-learning, SARSA variants)
– Each script instantiates SumoEnvironment + Agent + Model + logging
D. Persistence & Reporting
– models/ (.pth checkpoints)
– results/ (CSV training logs)
– outputs/ (PDF/PNG plots + plot.py)
E. Documentation & CI/CD
– docs/ (Sphinx sources, conf.py, examples, API md files)
– .github/workflows/ (build-publish, deploy-docs, tests, pre-commit)
– setup.py, pyproject.toml, requirements.txt
Relationships and Data Flow
1. experiments/*.py → import sumo_rl.environment & agents → instantiate SumoEnvironment
2. SumoEnvironment calls SUMO via TraCI:
env.reset() → TraCI launches sim → returns initial observation
env.step(action) → TrafficSignal sets phase → TraCI advances sim → returns next obs & reward
3. Agent receives observation → Model (DCRNN/Transformer/Q-table) computes action → back to env.step()
4. After each episode or epoch:
– metrics logged to results/*.csv
– model weights saved to models/*.pth
– plots generated via outputs/plot.py → outputs/*.pdf
5. docs/ updated → CI builds and deploys Sphinx site → badge updates
Architectural Patterns & Design Principles
• Modular Plugin Architecture: easily swap ObservationFunction, RewardFunction, Agent, Model
• Separation of Concerns:
• Environment layer (sumo_rl/environment)
• Agent layer (sumo_rl/agents)
• Model layer (sumo_rl/models)
• Data definitions (sumo_rl/nets)
• Graph-based RL: explicit pre-processing in util, DCRNN & transformer graph encoders/decoders
• Experiment-Driven Development: scripts in experiments/ form the entry points
• CI/CD for Quality: automated tests, pre-commit hooks, docs deployment
Detail to Capture
• Class names (SumoEnvironment, TrafficSignal, base_model, dcrnn_cell, transformer_model)
• File-level patterns (nets/ configurations, experiment scripts)
• Data artifacts and formats (.pth, .csv, .pdf)
• CI workflows (build, test, deploy-docs)