How vita-epfl/unitraj works
Project type and purpose
• UniTraj is a Python-based research framework for vehicle trajectory prediction.
• It unifies data loading (nuScenes, Waymo, Argoverse2, etc.), model definition (Autobot, MTR, Wayformer), training, evaluation, and logging under one Hydra-configured pipeline.
• Core use case: train & evaluate multiple trajectory prediction models on large real-world datasets.
File structure
• Top-level scripts:
• train.py – entry point for training.
• evaluation.py – entry point for evaluation.
• data_analysis.py – exploratory data analysis.
• Configuration: unitraj/configs/ (config.yaml + per-method YAMLs) → Hydra manages hierarchical configs.
• Dataset layer: unitraj/datasets/ – base_dataset.py + model-specific dataset classes.
• Model layer: unitraj/models/ –
• base_model/ – abstract base LightningModule, evaluation utilities, model utils, plus built-in eval configs.
• subfolders autobot/, mtr/, wayformer/ – method implementations, utilities, custom ops (C++/CUDA for MTR).
• Utilities: unitraj/utils/ – helper functions (config loader, TSNE, visualization).
• Data samples: unitraj/data_samples/ – example nuScenes pickled files.
• Third-party dependencies and build: setup.py, requirements.txt.
Readme insights
• Depicts a high-level “framework.png” showing data flow: ScenarioNet → UniTraj pipeline → model → metrics → WandB.
• Highlights Hydra, PyTorch-Lightning, WandB integration.
• Mentions optional evaluation tools (nuscenes-devkit, waymo-open-dataset).
Main components
a. Data Ingestion
• ScenarioNet data source (external service) → raw scenario files.
• local cache in unitraj/data_samples or H5 cache.
b. Configuration Service
• Hydra config manager (unitraj/configs/config.yaml + method/*.yaml).
c. Dataset Loader
• BaseDataset → method-specific dataset classes → PyTorch DataLoader.
d. Model Trainer
• PyTorch-Lightning Trainer orchestrates training loops, calls model modules.
• BaseModel defines common training, validation, logging hooks.
• Method modules (Autobot, MTR, Wayformer) implement forward, loss, inference.
• Custom ops (C++/CUDA) for performance in MTR.
e. Logging & Monitoring
• WandB (external service) for metrics, visualizations.
f. Evaluation
• evaluation.py orchestrates model checkpoint loading, running inference, computing metrics with nuscenes-devkit / waymo toolkits.
g. Utilities
• Visualization, TSNE, config helper functions.
Relationships and interactions
• ScenarioNet → Dataset Loader
• Hydra → Dataset Loader, Model Trainer, Evaluation scripts
• Dataset Loader → Lightning Trainer
• Lightning Trainer ↔ Model modules
• Trainer → WandB (logging)
• Evaluation script → Model checkpoint + Dataset → Evaluation metrics
• Utils are side-services called by Dataset, Model, or Training.
Architectural patterns and principles
• Plugin architecture: new models/datasets hook into base classes.
• Inheritance & polymorphism: base_dataset.py and base_model.py are extended.
• Configuration as code via Hydra: decouples code from hyperparameters.
• Modularization: clear separation of data, model, training, evaluation, utils.
Technologies and frameworks
• Hydra (config management)
• PyTorch-Lightning (training orchestration)
• PyTorch + custom C++/CUDA ops (compute)
• WandB (experiment tracking)
• ScenarioNet, nuscenes-devkit, waymo-open-dataset (data)
• Python build: setuptools (setup.py)
Detailed capture of essential elements
• Include:
• unitraj/train.py and train loop.
• unitraj/configs Hydra tree.
• Dataset classes hierarchy.
• Model classes hierarchy and custom ops.
• WandB integration hook.
• evaluation.py + external eval toolkit connections.
• Plugin points for new models/datasets.
• Control flow: CLI invocation → Hydra instantiation → object creation → training/eval.