# salmank255/road_waymo_baseline architecture

> Architecture diagram of the GitHub repository salmank255/road_waymo_baseline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/salmank255/road_waymo_baseline
- Repository: https://github.com/salmank255/road_waymo_baseline

## Overview

##### Project type and purpose
- This is a research-oriented deep-learning codebase implementing “3D-RetinaNet,” a single-stage video-action detection network.
- It supports training and evaluation on multiple datasets (ROAD-Waymo, ROAD, UCF-24).
- It provides scripts for data preprocessing, model training, inference (frame-level detection + tube building), and evaluation.

A. Configuration Manager
– “configs/” folder contains YAMLs for different datasets & model variants.
– Drives hyperparameters, dataset splits, model architecture choices, paths (datasets, weights, checkpoints).
B. Data Ingestion & Preprocessing
– Data sources:
- ROAD-Waymo, ROAD, UCF-24 raw video + annotations
- Kinetics-400 pre-trained weights (kinetics-pt/)
– Components:
- data/datasets.py, inference_dataset.py, nuscene_loader.py
- data/transforms.py (frame extraction, augmentations)
- split CSVs (weather_trainval.csv, location_trainval.csv, etc.)
– Output: Torch Datasets & DataLoaders yielding batched video clips + labels.
C. Model Definition
– models/backbones/slowfast.py (SlowFast or I3D)
– models/blocks.py, resnetFPN.py, nonlocal_helper.py
– models/necks/basic.py, utils.py
– models/heads/acar.py, linear.py (classification & regression heads)
– modules (anchor_box_kmeans.py, box_utils.py, detection_loss.py, solver.py)
D. Training Engine
– main.py (parses args & configs, constructs DataLoaders & Model, sets up optimizer & scheduler via modules/solver.py)
– train.py (epoch loop, forward, backward, checkpointing, tensorboardX logging)
– val.py (validation pass computing frame-mAP)
E. Inference & Tube Building
– inference.py / inference_vis.py (frame-level detection)
– gen_dets mode in main.py (generates detections, calls tubes.py)
– tubes.py (links frame detections into spatio-temporal action tubes)
F. Evaluation & Analysis
– modules/evaluation.py (frame & video mAP computation)
– extras/ (plot_stats.py, tally_results.py, classwise_counts, LaTeX tables)
– utils.py, json_merge.py (utility functions)
G. Utilities & Scripts
– add_location_labels.py, add_weather_labels.py, kmean_anchors.py, gt_vis.py, labels_count.py, etc.
– commands.sh, cmnd.txt for experiment management

##### Relationships & Data Flow
- Configuration Manager ──► Training Engine
(feeds hyperparameters & file paths)
- Data Ingestion ──► DataLoader ──► Training Engine
(video frames + labels batches)
- Pre-trained Weights ──► Model Definition ──► Training Engine
- Training Engine ──► Checkpoints + Logs
- Trained Model + Data Ingestion ──► Inference Engine
- Inference Engine ──► Tube Builder ──► Detections JSON
- Detections JSON + Ground-Truth ──► Evaluation Module ──► Metrics & Reports
- Extras ──► Plots & LaTeX (postprocessing for papers)

##### Architectural patterns & design principles
- Modular design: clear separation of data, model, training, inference, evaluation.
- Config-driven: decouple code from hyperparameters & dataset variants using YAMLs.
- Single-stage detection: end-to-end differentiable network with anchor-based heads.
- Pipeline stages: preprocessing → training/inference → postprocessing → evaluation.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Manager
    subgraph "Configuration Manager"
        CM["Configuration Manager"]:::disk
    end

    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing"
        DS["datasets.py"]:::disk
        ID["inference_dataset.py"]:::disk
        NL["nuscene_loader.py"]:::disk
        TF["transforms.py"]:::disk
        WTV["weather_trainval.csv"]:::disk
        WTE["weather_test.csv"]:::disk
        LTV["location_trainval.csv"]:::disk
        LTE["location_test.csv"]:::disk
    end

    %% Model Definition
    subgraph "Model Definition"
        subgraph "Backbones & Blocks"
            SF["slowfast.py"]:::runtime
            BLK["blocks.py"]:::runtime
            RFPN["resnetFPN.py"]:::runtime
            NLH["nonlocal_helper.py"]:::runtime
        end
        subgraph "Necks & Utils"
            NB["basic.py"]:::runtime
            NU["utils.py"]:::runtime
        end
        subgraph "Heads"
            AC["acar.py"]:::runtime
            LI["linear.py"]:::runtime
        end
        RN["retinanet.py"]:::runtime
    end

    %% Anchor & Utility Modules
    subgraph "Anchor & Utility Modules"
        AK1["anchor_box_kmeans.py"]:::runtime
        AK2["anchor_box_retinanet.py"]:::runtime
        BU["box_utils.py"]:::runtime
        DL["detection_loss.py"]:::runtime
        MU["utils.py"]:::runtime
    end

    %% Training Engine
    subgraph "Training Engine"
        MP["main.py"]:::runtime
        TR["train.py"]:::runtime
        VL["val.py"]:::runtime
        SL["solver.py"]:::runtime
    end

    %% Inference & Tube Building
    subgraph "Inference & Tube Building"
        INF["inference.py"]:::runtime
        IV["inference_vis.py"]:::runtime
        GD["gen_dets.py"]:::runtime
        TB["tubes.py"]:::runtime
        TH["tube_helper.py"]:::runtime
    end

    %% Evaluation & Analysis
    subgraph "Evaluation & Analysis"
        EM["evaluation.py"]:::post
        PS["plot_stats.py"]:::post
        TRL["tally_results.py"]:::post
        CW["classwise_counts.tex"]:::post
        R1["results_road.tex"]:::post
        R2["results_ucf24.tex"]:::post
        R3["results_classwise_road.tex"]:::post
    end

    %% Utilities & Scripts
    subgraph "Utilities & Scripts"
        AL["add_location_labels.py"]:::disk
        AW["add_weather_labels.py"]:::disk
        KA["kmean_anchors.py"]:::disk
        GV["gt_vis.py"]:::disk
        LC["labels_count.py"]:::disk
        LCC["labels_count_cities.py"]:::disk
        JM["json_merge.py"]:::disk
        CS["commands.sh"]:::disk
        CT["cmnd.txt"]:::disk
    end

    %% Data Flows
    CM -->|"configs & hyperparams"| MP
    DS -->|"Torch Dataset"| MP
    ID -->|"Inference Dataset"| INF
    NL -->|"Data Loader"| MP
    TF -->|"Transforms"| MP
    MP -->|"init Model"| SF
    MP -->|"init Model"| BLK
    MP -->|"init Model"| RFPN
    MP -->|"init Model"| NLH
    MP -->|"init Model"| NB
    MP -->|"init Model"| NU
    MP -->|"init Model"| AC
    MP -->|"init Model"| LI
    MP -->|"init Model"| RN
    MP -->|"optimizer & scheduler"| SL
    TR -->|"training loop"| SL
    TR -->|"checkpoint"| MP
    TR -->|"logs"| MP
    VL -->|"validation"| MP
    MP -->|"trained weights"| INF
    INF -->|"frame detections"| TB
    GD -->|"generate detections"| TB
    TB -->|"detections.json"| EM
    EM -->|"metrics"| PS
    EM -->|"metrics"| TRL
    PS -->|"plots"| CW
    TRL -->|"tables"| R1
    TRL -->|"tables"| R2
    TRL -->|"tables"| R3
    AL -->|"label scripts"| DS
    AW -->|"label scripts"| DS
    KA -->|"anchors"| AK1
    GV -->|"visualizations"| EM
    LC -->|"counts"| EM
    LCC -->|"counts"| EM
    JM -->|"merge JSON"| EM
    CS -->|"experiment cmds"| TR
    CT -->|"experiment cmds"| TR

    %% Click Events
    click CM "https://github.com/salmank255/road_waymo_baseline/tree/master/configs/"
    click DS "https://github.com/salmank255/road_waymo_baseline/blob/master/data/datasets.py"
    click ID "https://github.com/salmank255/road_waymo_baseline/blob/master/data/inference_dataset.py"
    click NL "https://github.com/salmank255/road_waymo_baseline/blob/master/data/nuscene_loader.py"
    click TF "https://github.com/salmank255/road_waymo_baseline/blob/master/data/transforms.py"
    click WTV "https://github.com/salmank255/road_waymo_baseline/blob/master/weather_trainval.csv"
    click WTE "https://github.com/salmank255/road_waymo_baseline/blob/master/weather_test.csv"
    click LTV "https://github.com/salmank255/road_waymo_baseline/blob/master/location_trainval.csv"
    click LTE "https://github.com/salmank255/road_waymo_baseline/blob/master/location_test.csv"
    click SF "https://github.com/salmank255/road_waymo_baseline/blob/master/models/backbones/slowfast.py"
    click BLK "https://github.com/salmank255/road_waymo_baseline/blob/master/models/blocks.py"
    click RFPN "https://github.com/salmank255/road_waymo_baseline/blob/master/models/resnetFPN.py"
    click NLH "https://github.com/salmank255/road_waymo_baseline/blob/master/models/nonlocal_helper.py"
    click NB "https://github.com/salmank255/road_waymo_baseline/blob/master/models/necks/basic.py"
    click NU "https://github.com/salmank255/road_waymo_baseline/blob/master/models/necks/utils.py"
    click AC "https://github.com/salmank255/road_waymo_baseline/blob/master/models/heads/acar.py"
    click LI "https://github.com/salmank255/road_waymo_baseline/blob/master/models/heads/linear.py"
    click RN "https://github.com/salmank255/road_waymo_baseline/blob/master/models/retinanet.py"
    click AK1 "https://github.com/salmank255/road_waymo_baseline/blob/master/modules/anchor_box_kmeans.py"
    click AK2 "https://github.com/salmank255/road_waymo_baseline/blob/master/modules/anchor_box_retinanet.py"
    click BU "https://github.com/salmank255/road_waymo_baseline/blob/master/modules/box_utils.py"
    click DL "https://github.com/salmank255/road_waymo_baseline/blob/master/modules/detection_loss.py"
    click MU "https://github.com/salmank255/road_waymo_baseline/blob/master/modules/utils.py"
    click MP "https://github.com/salmank255/road_waymo_baseline/blob/master/main.py"
    click TR "https://github.com/salmank255/road_waymo_baseline/blob/master/train.py"
    click VL "https://github.com/salmank255/road_waymo_baseline/blob/master/val.py"
    click SL "https://github.com/salmank255/road_waymo_baseline/blob/master/modules/solver.py"
    click INF "https://github.com/salmank255/road_waymo_baseline/blob/master/inference.py"
    click IV "https://github.com/salmank255/road_waymo_baseline/blob/master/inference_vis.py"
    click GD "https://github.com/salmank255/road_waymo_baseline/blob/master/gen_dets.py"
    click TB "https://github.com/salmank255/road_waymo_baseline/blob/master/tubes.py"
    click TH "https://github.com/salmank255/road_waymo_baseline/blob/master/modules/tube_helper.py"
    click EM "https://github.com/salmank255/road_waymo_baseline/blob/master/modules/evaluation.py"
    click PS "https://github.com/salmank255/road_waymo_baseline/blob/master/extras/plot_stats.py"
    click TRL "https://github.com/salmank255/road_waymo_baseline/blob/master/extras/tally_results.py"
    click CW "https://github.com/salmank255/road_waymo_baseline/blob/master/extras/classwise_counts.tex"
    click R1 "https://github.com/salmank255/road_waymo_baseline/blob/master/extras/results_road.tex"
    click R2 "https://github.com/salmank255/road_waymo_baseline/blob/master/extras/results_ucf24.tex"
    click R3 "https://github.com/salmank255/road_waymo_baseline/blob/master/extras/results_classwise_road.tex"
    click AL "https://github.com/salmank255/road_waymo_baseline/blob/master/add_location_labels.py"
    click AW "https://github.com/salmank255/road_waymo_baseline/blob/master/add_weather_labels.py"
    click KA "https://github.com/salmank255/road_waymo_baseline/blob/master/kmean_anchors.py"
    click GV "https://github.com/salmank255/road_waymo_baseline/blob/master/gt_vis.py"
    click LC "https://github.com/salmank255/road_waymo_baseline/blob/master/labels_count.py"
    click LCC "https://github.com/salmank255/road_waymo_baseline/blob/master/labels_count_cities.py"
    click JM "https://github.com/salmank255/road_waymo_baseline/blob/master/json_merge.py"
    click CS "https://github.com/salmank255/road_waymo_baseline/blob/master/commands.sh"
    click CT "https://github.com/salmank255/road_waymo_baseline/blob/master/cmnd.txt"

    %% Styles
    classDef disk fill:#E0F7FA,stroke:#006064
    classDef runtime fill:#E8F5E9,stroke:#1B5E20
    classDef post fill:#FFF3E0,stroke:#E65100
```
