How sahityabiswas/cityflow works
Purpose and Entry Point
This is a local Python reinforcement-learning simulator for adaptive control of a single four-way traffic intersection. Running main.py starts training and visualization in a PyGame window. The executable path is main.py, which delegates the simulation lifecycle to src/simulation.py.
Runtime Composition
The system runs as one in-process Python application, with:
• PyGame for the interactive simulation, frame rendering, and HUD.
• PyTorch for the Deep Q-Network (DQN) and optimization.
• No server, database, cloud infrastructure, or external runtime service is indicated.
Dependencies are declared in requirements.txt; core modules reside under src/.
Configuration Boundary
src/config.py centralizes simulation layout/constants and RL hyperparameters, including learning rate, discount factor, batch size, replay capacity, and epsilon schedule. This module is the shared policy/configuration boundary consumed by the environment, agent, model, and simulation:
• src/config.py
• src/simulation.py
• src/agent.py
Traffic Environment and State Transition
src/environment.py owns the traffic-domain simulation: lane queues, vehicle spawning and movement, collision-related checks, signal application, state extraction, and reward calculation. Its state is a four-value lane-queue observation (north, south, east, west). An action selects one of six permitted traffic-light phases. The environment advances vehicles, handles emergency-vehicle priority, and returns the reward signal based on queue and wait-time reduction:
• src/environment.py
• src/config.py
DQN Model
src/model.py defines the PyTorch Q-network used to estimate action values from the four-dimensional traffic state. It is intentionally separated from learning policy and simulation concerns: the model supplies forward inference and trainable parameters, while the agent determines exploration and invokes optimization:
• src/model.py
• src/agent.py
Learning Agent and Replay Boundary
src/agent.py implements the reinforcement-learning controller. It performs epsilon-greedy selection between random phases and Q-network exploitation, stores (state, action, reward, next_state) transitions in a replay buffer, samples batches after sufficient experience exists, updates the DQN, and decays epsilon. This is the boundary between environment feedback and model training:
• src/agent.py
• src/model.py
• src/environment.py
Simulation Orchestration and Control Flow
src/simulation.py coordinates repeated simulated “days”: initialize PyGame/environment/agent, spawn and progress traffic, obtain state, request an action, apply the phase, step the environment, record the transition, train the agent, and render the intersection plus training metrics. Day-level reward and waiting-time statistics feed the live learning curves and HUD. Emergency detection may override normal selected signaling before reward and replay storage:
• src/simulation.py
• src/environment.py
• src/agent.py
Supporting Artifacts
notebooks/SCTLC.ipynb is a reference/exploratory notebook rather than the application entry path. docs/Report_Team_SSP.pdf contains project research documentation. The runnable architecture is concentrated in main.py and the five core src/ modules.
Main entry points: agent.py DQN Agent · simulation.py Simulation Orchestrator · environment.py Traffic Environment · config.py Simulation & RL Config
Components 13
Interactive Runtime
- Application Entry · Starts training and visualization.
main.py - Simulation Orchestrator · Runs day cycles, control flow, metrics, and rendering.
src/simulation.py - PyGame Window & HUD · Renders the intersection and live learning metrics.
DQN Learning
- DQN Agent · Selects actions, records experience, trains, and decays epsilon.
src/agent.py - Replay Buffer · Stores and batches state-transition tuples.
- PyTorch Q-Network · Estimates six action values from traffic state.
src/model.py
Traffic Domain
- Traffic Environment · Owns lanes, vehicles, signals, state transitions, and rewards.
src/environment.py - Lane Queue State · North, south, east, and west queue values.
- Traffic Phase Control · Applies permitted signal phases with emergency priority.
- Queue & Wait Reward · Rewards queue and wait-time reduction.
Other components
- Simulation & RL Config · Shared layout constants and learning hyperparameters.
src/config.py - Exploratory Notebook · Reference and exploratory work outside the runtime path.
notebooks/SCTLC.ipynb - Project Report
docs/Report_Team_SSP.pdf
Connections 17
- Application Entry → Simulation Orchestrator: starts
- Simulation Orchestrator → PyGame Window & HUD: renders
- Simulation Orchestrator → Traffic Environment: coordinates steps
- Simulation Orchestrator → DQN Agent: requests action / trains
- Traffic Environment → Lane Queue State: extracts
- Lane Queue State → DQN Agent: observation
- DQN Agent → Traffic Phase Control: selects phase
- Traffic Phase Control → Traffic Environment: applies signals
- Traffic Environment → Queue & Wait Reward: calculates
- Queue & Wait Reward → DQN Agent: feedback
- DQN Agent → Replay Buffer: stores transitions
- Replay Buffer → DQN Agent: samples batches
- DQN Agent → PyTorch Q-Network: optimizes
- PyTorch Q-Network → DQN Agent: Q-values
- Simulation & RL Config → Simulation Orchestrator: runtime settings
- Simulation & RL Config → Traffic Environment: layout/constants
- Simulation & RL Config → DQN Agent: RL hyperparameters