How alniyatrui/cafe-ad works
Project Type & Purpose
• This is a research‐grade full‐stack ML system for autonomous‐vehicle trajectory planning (CAFE-AD), built atop the PLUTO framework.
• Key goals: ingest nuPlan scenarios, preprocess & augment data, train a transformer‐based planner (PLUTO model) with cross‐scenario adaptive features, then run inference in simulation and evaluate trajectories.
High-Level Components
a. Data & Config
• External: nuPlan dataset (raw scenarios), HuggingFace 100K split
• Config layer: YAML files under config/ (trainer, model, planner, augmentation, filters, training, lightning)
• Scripts: shell wrappers in script/ + train.sh, simulation.sh
b. Preprocessing & Augmentation
• preprocess/preprocess.py: scenario distribution & token list generation
• src/data_augmentation/contrastive_scenario_generator.py + config-driven augmentation
c. Feature Extraction
• src/feature_builders/…: map, agent, scenario features → unified feature vectors
• src/features/pluto_feature.py: final feature representation
d. Training Pipeline
• DataModule & builder: src/custom_training/custom_datamodule.py, custom_training_builder.py
• Model & loss: src/models/pluto/… (encoder modules, transformer, embeddings, MLP layers, collision loss)
• Optimizer & scheduler: src/optim/warmup_cos_lr.py
• Trainer: PyTorch Lightning wrapper in src/models/pluto/pluto_trainer.py
• Entry point: run_training.py driven by train.sh
e. Inference & Simulation
• Planners: src/planners/pluto_planner*.py (learned, IDM, log‐replay variants)
• Scenario manager: src/scenario_manager/ (route, occupancy, cost maps)
• Runner: run_simulation.py, idm_simulation.py, log_replay.py, orchestrated by simulation.sh
f. Post-processing & Evaluation
• Forward sim: src/post_processing/forward_simulation/…
• Metrics & evaluation: src/post_processing/evaluation/comfort_metrics.py, src/metrics/*.py
• Visualization & results: result.py, src/utils/vis.py
Interactions & Data Flow
1. Raw nuPlan Data → Preprocess → token_list, filtered scenarios
2. Filtered scenarios + Augmentation → Feature Builders → Batched tensors
3. DataLoader → PLUTO Model (encoders → transformer → decoder) → Loss functions → Trainer → Checkpoint
4. Checkpoint + Config → Planner component → Simulation Loop (scenario_manager drives environment)
5. Trajectory outputs → Post-processing simulators & metrics → Results & visualizations
Architectural Patterns & Principles
• Modular pipelines: separate dirs for preprocess, augment, features, model, planners, post_processing
• Config‐driven orchestration: YAML + Hydra-style parameters
• Pluggable components: multiple planner backends (learned vs. heuristic)
• Clear separation of concerns: DataModule vs. Model vs. Trainer vs. Inference vs. Evaluation