# aharley/simple_bev architecture

> Architecture diagram of the GitHub repository aharley/simple_bev, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aharley/simple_bev
- Repository: https://github.com/aharley/simple_bev

## Overview

##### Project Type and Purpose
- Type: Research codebase for multi-sensor BEV (Bird’s-Eye-View) perception.
- Purpose: Train and evaluate deep-learning models that fuse camera and radar data into a BEV representation for perception tasks.

##### File Structure
Top-level components:
- Scripts and entry points
– train_nuscenes.py, train_lyft.py, eval_nuscenes.py, eval_lyft.py
– shell wrappers (train.sh, eval.sh, *_runner.sh, get_*_model.sh)
- Model definitions (nets/)
– bevformernet.py, liftnet.py (core architectures)
– ops/ (custom CUDA/CPU extensions for multi-scale deformable attention)
- Data loader
– nuscenesdataset.py (reads nuScenes data, applies augmentations)
- Utilities (utils/)
– basic I/O, geometric transforms, image processing, voxelization
- Checkpointing and logging
– saverloader.py
– log directories, checkpoints directory (via command-line flags)
- Visualization
– vis_nuscenes.py, vis_collage.py

##### Extract Insights from the README
- Training workflow: load data → preprocess → feed to model → compute loss/backprop → save checkpoints → log metrics
- Evaluation workflow: load checkpoint → load data → model inference → compute metrics (mean IoU) → save logs
- Tensor conventions: B,S,C,Z,Y,X axes; geometry transforms (cam_T_world, etc.)
- External dependencies: PyTorch, CUDA extensions, nuScenes dataset

##### Main Components and Their Relationships
A. Data Sources
- nuScenes raw data (images, RADAR sweeps, annotations)
B. Data Loader & Preprocessor
- nuscenesdataset.py → uses utils/geom.py, utils/improc.py, utils/vox.py
C. Model Core
1. Backbone Encoder (ResNet-based image encoders)
2. View Transformer / LiftNet (lift image features into voxel space)
3. BEVFormerNet (fusion of multi-sensor features in BEV)
4. Segmentation Head (predict BEV segmentation map)
D. Custom Ops Library
- nets/ops modules (ms_deform_attn) implemented in C++/CUDA → compiled via setup.py
E. Training Manager
- train_nuscenes.py orchestrates data loader, model instantiation, optimizer/scheduler, loss, backprop, saverloader
F. Evaluation Manager
- eval_nuscenes.py loads model checkpoint, runs inference, computes metrics
G. Checkpointing & Logging
- saverloader.py for saving/loading model states
- Command-line flags define log_dir, ckpt_dir; logs written to filesystem
H. Visualization Tools
- vis_nuscenes.py for qualitative outputs (collages, GIFs)

##### Architectural Patterns & Design Principles
- Modular separation: data loading, preprocessing, model definition, training, evaluation
- Custom C++/CUDA ops for performance-critical attention modules
- Clear tensor conventions for reproducibility
- Command-line configurability for experiments

– Blue for data components (loader, dataset)
– Green for model components (encoders, BEVFormerNet)
– Orange for scripts/orchestration (train, eval)
– Gray for utilities and custom ops

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data"
        DS[/"nuScenes Raw Data"/]:::data
        DL["Data Loader & Preprocessor"]:::data
    end
    DS --> DL
    subgraph "Utilities"
        Geom["geom.py"]:::util
        Improc["improc.py"]:::util
        Vox["vox.py"]:::util
    end
    DL --> Geom
    DL --> Improc
    DL --> Vox

    subgraph "Model Core"
        BE["Backbone Encoder"]:::model
        RE["Radar Encoder (optional)"]:::model
        VT["View Transformer / LiftNet"]:::model
        MF["BEVFormerNet (Fusion)"]:::model
        SH["Segmentation Head"]:::model
    end
    DL --> BE
    DL -.-> RE
    BE --> VT
    RE -.-> VT
    VT --> MF
    MF --> SH

    subgraph "Custom Ops"
        MFDAF["ms_deform_attn_func.py"]:::util
        MFDAM["ms_deform_attn.py"]:::util
        Setup["setup.py"]:::util
        Make["make.sh"]:::util
        CPU["ms_deform_attn_cpu.cpp/.h"]:::util
        CUDA["ms_deform_attn_cuda.cu/.h"]:::util
        Common["ms_deform_attn.h, vision.cpp"]:::util
    end
    MF -->|invokes| MFDAF
    MF -->|invokes| MFDAM
    MFDAF --> CPU
    MFDAF --> CUDA
    MFDAF --> Common
    MFDAM --> Setup
    MFDAM --> Make

    subgraph "Orchestration"
        TM["Training Manager"]:::orch
        EM["Evaluation Manager"]:::orch
        CK["Checkpointing & Logging"]:::orch
    end
    DL --> TM
    TM --> BE
    TM --> MF
    TM --> CK
    EM --> DL
    EM --> BE
    EM --> MF
    EM --> CK

    subgraph "Visualization"
        VIS1["vis_nuscenes.py"]:::orch
        VIS2["vis_collage.py"]:::orch
    end
    SH --> VIS1
    SH --> VIS2

    classDef data fill:#ADD8E6,stroke:#000;
    classDef model fill:#90EE90,stroke:#000;
    classDef util fill:#D3D3D3,stroke:#000;
    classDef orch fill:#FFA500,stroke:#000;

    click DL "https://github.com/aharley/simple_bev/blob/main/nuscenesdataset.py"
    click Geom "https://github.com/aharley/simple_bev/blob/main/utils/geom.py"
    click Improc "https://github.com/aharley/simple_bev/blob/main/utils/improc.py"
    click Vox "https://github.com/aharley/simple_bev/blob/main/utils/vox.py"
    click BE "https://github.com/aharley/simple_bev/blob/main/nets/bevformernet.py"
    click VT "https://github.com/aharley/simple_bev/blob/main/nets/liftnet.py"
    click SH "https://github.com/aharley/simple_bev/blob/main/nets/segnet.py"
    click MFDAF "https://github.com/aharley/simple_bev/blob/main/nets/ops/functions/ms_deform_attn_func.py"
    click MFDAM "https://github.com/aharley/simple_bev/blob/main/nets/ops/modules/ms_deform_attn.py"
    click Setup "https://github.com/aharley/simple_bev/blob/main/nets/ops/setup.py"
    click Make "https://github.com/aharley/simple_bev/blob/main/nets/ops/make.sh"
    click CPU "https://github.com/aharley/simple_bev/blob/main/nets/ops/src/cpu/ms_deform_attn_cpu.cpp"
    click CUDA "https://github.com/aharley/simple_bev/blob/main/nets/ops/src/cuda/ms_deform_attn_cuda.cu"
    click Common "https://github.com/aharley/simple_bev/blob/main/nets/ops/src/ms_deform_attn.h"
    click TM "https://github.com/aharley/simple_bev/blob/main/train_nuscenes.py"
    click EM "https://github.com/aharley/simple_bev/blob/main/eval_nuscenes.py"
    click CK "https://github.com/aharley/simple_bev/blob/main/saverloader.py"
    click VIS1 "https://github.com/aharley/simple_bev/blob/main/vis_nuscenes.py"
    click VIS2 "https://github.com/aharley/simple_bev/blob/main/vis_collage.py"
```
