How salmank255/road_waymo_baseline works
Project type and purpose
• This is a research-oriented deep-learning codebase implementing “3D-RetinaNet,” a single-stage video-action detection network.
• It supports training and evaluation on multiple datasets (ROAD-Waymo, ROAD, UCF-24).
• It provides scripts for data preprocessing, model training, inference (frame-level detection + tube building), and evaluation.
A. Configuration Manager
– “configs/” folder contains YAMLs for different datasets & model variants.
– Drives hyperparameters, dataset splits, model architecture choices, paths (datasets, weights, checkpoints).
B. Data Ingestion & Preprocessing
– Data sources:
• ROAD-Waymo, ROAD, UCF-24 raw video + annotations
• Kinetics-400 pre-trained weights (kinetics-pt/)
– Components:
• data/datasets.py, inference_dataset.py, nuscene_loader.py
• data/transforms.py (frame extraction, augmentations)
• split CSVs (weather_trainval.csv, location_trainval.csv, etc.)
– Output: Torch Datasets & DataLoaders yielding batched video clips + labels.
C. Model Definition
– models/backbones/slowfast.py (SlowFast or I3D)
– models/blocks.py, resnetFPN.py, nonlocal_helper.py
– models/necks/basic.py, utils.py
– models/heads/acar.py, linear.py (classification & regression heads)
– modules (anchor_box_kmeans.py, box_utils.py, detection_loss.py, solver.py)
D. Training Engine
– main.py (parses args & configs, constructs DataLoaders & Model, sets up optimizer & scheduler via modules/solver.py)
– train.py (epoch loop, forward, backward, checkpointing, tensorboardX logging)
– val.py (validation pass computing frame-mAP)
E. Inference & Tube Building
– inference.py / inference_vis.py (frame-level detection)
– gen_dets mode in main.py (generates detections, calls tubes.py)
– tubes.py (links frame detections into spatio-temporal action tubes)
F. Evaluation & Analysis
– modules/evaluation.py (frame & video mAP computation)
– extras/ (plot_stats.py, tally_results.py, classwise_counts, LaTeX tables)
– utils.py, json_merge.py (utility functions)
G. Utilities & Scripts
– add_location_labels.py, add_weather_labels.py, kmean_anchors.py, gt_vis.py, labels_count.py, etc.
– commands.sh, cmnd.txt for experiment management
Relationships & Data Flow
• Configuration Manager ──► Training Engine
(feeds hyperparameters & file paths)
• Data Ingestion ──► DataLoader ──► Training Engine
(video frames + labels batches)
• Pre-trained Weights ──► Model Definition ──► Training Engine
• Training Engine ──► Checkpoints + Logs
• Trained Model + Data Ingestion ──► Inference Engine
• Inference Engine ──► Tube Builder ──► Detections JSON
• Detections JSON + Ground-Truth ──► Evaluation Module ──► Metrics & Reports
• Extras ──► Plots & LaTeX (postprocessing for papers)
Architectural patterns & design principles
• Modular design: clear separation of data, model, training, inference, evaluation.
• Config-driven: decouple code from hyperparameters & dataset variants using YAMLs.
• Single-stage detection: end-to-end differentiable network with anchor-based heads.
• Pipeline stages: preprocessing → training/inference → postprocessing → evaluation.