How hustvl/diffusiondrive works
Project Type and Purpose
• DiffusionDrive is an open-source research framework for real-time end-to-end autonomous driving, built on top of the NAVSIM simulator.
• It provides data-download utilities, scenario building, planning/simulation modules, a truncated diffusion agent, training/evaluation pipelines, and visualization tools.
Main Architectural Components
A. Data & Environment Layer
• download/ – shell scripts to fetch NAVSIM maps, scenarios, train/test splits
• NAVSIM simulator (external dependency)
B. Scenario & Data Preparation
• navsim/planning/scenario_builder – constructs driving scenarios from map and traffic data
• navsim/common/dataloader – PyTorch Dataset + DataLoader abstractions
• navsim/planning/script/run_dataset_caching.py – caches precomputed metrics for faster iteration
C. Core Agent & Model
• navsim/agents/abstract_agent.py – base agent interface (plugin point)
• navsim/agents/diffusiondrive – truncated diffusion model implementation (backbone, scheduler, loss modules, training callbacks)
• navsim/agents/transfuser & others – reference implementations for alternative policies
D. Planning & Simulation
• navsim/planning/simulation/planner/pdm_planner – runs closed-loop planning with PDM (proposal, scoring, simulation)
• navsim/planning/training – LightningModule wrappers, feature/target builders, callbacks
• navsim/planning/utils/multithreading – optional Ray or thread-pool parallel execution of rollouts
E. Training & Evaluation Orchestration
• docs/train_eval.md – high-level instructions
• navsim/planning/script/run_training.py – config-driven training entrypoint
• navsim/planning/script/run_pdm_score.py – evaluate model in closed-loop, compute PDMS
• scripts/training & scripts/evaluation – convenience shell scripts
F. Visualization & Analysis
• navsim/visualization – BEV, camera, LiDAR renderers and plotting utilities
• tutorial/tutorial_visualization.ipynb – example notebook
Relationships & Data Flow
1. Download & Prepare Data →
2. Scenario Builder loads map/scene definitions →
3. DataLoader yields sensor + map + ego/human trajectories →
4. Training loop (LightningModule) invokes:
• backbone (ResNet-34/50) for perception features
• truncated diffusion denoiser for control outputs
• loss calculation, backprop, checkpointing
5. After training, run PDM scoring pipeline:
• load trained weights → simulation planner → agent takes actions in NAVSIM → metrics collected → PDMS computed
6. Visualization tools consume trajectories + sensor data for qualitative analysis
Architectural Patterns & Design Principles
• Plugin architecture for “Agent” abstraction (easy to swap diffusion, transfuser, constant-velocity, human, MLP)
• Config-driven design (YAML files under navsim/planning/script/config)
• Modular separation: data ingestion, model, planning, evaluation, visualization
• Use of PyTorch Lightning for standardized training loops and callbacks
• Parallel execution via Ray or thread pools for scenario rollouts and metric caching