usail-hkust/llmtscs

1. What this repository is

LLMLight is a traffic-signal-control framework centered on a shared simulation environment. It compares three kinds of controllers: heuristic baselines, RL/DNN baselines, and LLM-based agents. The repo also contains a separate LightGPT fine-tuning pipeline for adapting an LLM to traffic signal decisions.

2. Core runtime boundary: simulation vs. decision policy

The main architectural split is between the traffic simulation layer and the control policy layer. The simulation layer is built around CityFlow and the road-network/traffic scenario JSON files under data/. The policy layer contains all agents that observe the current traffic state and emit a phase/action for each intersection.

3. Environment and state construction

utils/cityflow_env.py is the central environment wrapper. Supporting utilities in utils/ handle sampling, replay construction, update loops, evaluation, and transfer testing. The data/<city>/<scenario>/ folders provide the road network definition plus anonymized real/synthetic traffic traces that drive the simulator.

4. Agent hierarchy

models/agent.py is the base abstraction for controllers. Everything else in models/ is a concrete policy implementation: fixed-time, random, max-pressure variants, MPLight/CoLight/PressLight/AttendLight, plus advanced and efficient variants. This makes the model layer a plug-in set of interchangeable controllers over the same environment API.

5. Heuristic and RL baselines

The root run_*.py scripts are the execution entrypoints for benchmark runs. Scripts such as run_fixedtime.py, run_maxpressure.py, run_mplight.py, run_colight.py, and their advanced_ / efficient_ counterparts wire a specific agent class to the environment, execute training or evaluation, and produce logs/results for comparison.

6. LLM-based inference path

run_chatgpt.py, run_open_LLM.py, and run_open_LLM_with_vllm.py provide the LLM control path. Here, the simulator state is transformed into a textual prompt, sent to either a hosted model (ChatGPT) or a local open-source model, and the generated response is mapped back to a traffic-signal action. models/chatgpt.py holds the external API setup for OpenAI-style calls.

7. Prompting and instruction design

The prompts/ directory stores prompt templates and domain-knowledge material used to encode traffic state into natural-language instructions. models/chatgpt.py and the utilities around it sit at the boundary between structured simulator state and the LLM’s text interface.

8. LightGPT fine-tuning pipeline

The finetune/ package is a separate training subsystem for adapting a base LLM into LightGPT. run_imitation_finetune.py performs supervised imitation learning from collected traffic decision data, merge_lora.py merges LoRA adapters back into a base model, and run_policy_refinement.py performs critic-guided refinement after data collection. state_action_2_instructions.py converts simulator state/action pairs into instruction-style training examples.

9. Data collection for refinement

run_policy_refinement_data_collection.py bridges RL and LLM training: it uses a trained RL controller as a teacher to generate policy-refinement data, which is then stored under data/cgpr/. This creates a two-stage flow: simulator rollout from an RL agent, then instruction-formatted training data for the LLM adapter.

10. Evaluation and analysis utilities

utils/model_test.py, utils/RL_transfer_test.py, and related helpers support benchmarking and cross-scenario transfer. utils/llm_aft_trainer.py and ranking/loss utilities suggest additional training logic for alignment or preference-style optimization, but they remain within the same simulator-to-policy pipeline.

11. Frontend replay/visualization

frontend/page/ is a static browser-based viewer used to replay and inspect traffic-control behavior. It is not the control runtime itself; it consumes replay artifacts so users can visualize intersection states and agent decisions after an experiment.

12. Repository-level artifacts

The data/ tree is the main scenario asset store, while media/ holds figures and demo materials. The README also references model checkpoints and logs as external outputs of the runs. Architecturally, those artifacts are downstream products of the training/evaluation scripts rather than first-class application services.

13. Important external dependencies

CityFlow is the simulation engine and requires Linux. TensorFlow appears to support older RL components, while Hugging Face transformers, peft, accelerate, datasets, and optionally vllm power the LLM side. wandb is used for experiment tracking. These dependencies matter because they define two distinct execution modes: classic RL/simulation and LLM inference/fine-tuning.