# nesl/radar-camera-fusion-depth architecture

> Architecture diagram of the GitHub repository nesl/radar-camera-fusion-depth, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nesl/radar-camera-fusion-depth
- Repository: https://github.com/nesl/radar-camera-fusion-depth

## Overview

##### Project type and purpose
- This is a research‐oriented, two‐stage deep‐learning pipeline for depth estimation, fusing camera images and mmWave radar point clouds.
- The code handles dataset preparation (nuScenes), two model stages (RadarNet and FusionNet), training, evaluation, and visualization.

##### File structure
- Top‐level directories:
- bash/ – shell scripts for data setup, training, and evaluation
- setup/ – Python scripts to preprocess raw nuScenes data into training/test sets
- src/ – core code: dataset classes, transforms, model definitions (radarnet_model.py, fusionnet_model.py), training and inference entrypoints
- tools/ – visualization utilities
- Key patterns: a clear “data preparation → stage1 model → stage2 model → evaluation/visualization” pipeline
- Configuration: requirements.txt, license, and README guide installation and high‐level flow

##### README insights
- Describes sequence: environment setup → dataset prep → train RadarNet → dataset prep for FusionNet → train FusionNet → evaluate
- Emphasizes use of nuScenes, PyTorch, TensorBoard, and provided shell wrappers

## Diagram (Mermaid)

```mermaid
graph TD
    %% External Data
    RawData["nuScenes Raw Data"]:::external

    %% Data Preparation Layer
    subgraph "Data Preparation" 
        direction TB
        GenPanoptic["gen_panoptic_seg.py"]:::script
        SetupSplit["setup_dataset_nuscenes.py"]:::script
        SetupRadarIn["setup_dataset_nuscenes_radarnet.py"]:::script
        SetupRadarTestIn["setup_dataset_nuscenes_radarnet_test.py"]:::script
        SetupFinalTest["setup_dataset_nuscenes_test.py"]:::script
        DerivedData[(Preprocessed Data Store)]:::dataStore
    end

    %% Bash Orchestration for Data Prep
    subgraph "Bash: Data Prep" 
        direction TB
        BashSetup1["setup_dataset_nuscenes.sh"]:::bash
        BashSetup2["setup_dataset_nuscenes_radarnet.sh"]:::bash
        BashSetup3["setup_dataset_nuscenes_radarnet_test.sh"]:::bash
        BashSetup4["setup_dataset_nuscenes_test.sh"]:::bash
    end

    %% Stage 1: RadarNet
    subgraph "Stage 1: RadarNet" 
        direction TB
        RNModel["radarnet_model.py"]:::model
        RNTransforms["radarnet_transforms.py"]:::script
        RNTrainEntry["train_radarnet.py"]:::script
        RNTrainMain["radarnet_main.py"]:::script
        RNInferEntry["run_radarnet.py"]:::script
        RNEval["eval_stage_1_depth.py"]:::script
        RNSave["save_stage_1_depth.py"]:::script
        RNSaveUtils["save_stage_1_utils.py"]:::script
        RNCheckpoints[(RadarNet Checkpoints)]:::dataStore
    end

    %% Bash Orchestration for RadarNet
    subgraph "Bash: RadarNet" 
        direction TB
        BashTrainRN["train_radarnet_nuscenes.sh"]:::bash
        BashRunRN["run_radarnet_nuscenes.sh"]:::bash
    end

    %% Stage 2: FusionNet
    subgraph "Stage 2: FusionNet" 
        direction TB
        FNModel["fusionnet_model.py"]:::model
        FNLosses["fusionnet_losses.py"]:::script
        FNTransforms["fusionnet_transforms.py"]:::script
        FNTrainEntry["train_fusionnet.py"]:::script
        FNTrainMain["fusionnet_main.py"]:::script
        FNInferEntry["run_fusionnet.py"]:::script
        FNEvalUtils["eval_utils.py"]:::script
        FNSave["save_depth_radar.py"]:::script
        FNCheckpoints[(FusionNet Checkpoints)]:::dataStore
    end

    %% Bash Orchestration for FusionNet
    subgraph "Bash: FusionNet" 
        direction TB
        BashTrainFN["train_fusionnet_nuscenes.sh"]:::bash
        BashRunFN["run_fusionnet_nuscenes.sh"]:::bash
        BashRunFNTest["run_fusionnet_nuscenes_test.sh"]:::bash
    end

    %% Monitoring & Logging
    LogUtils["log_utils.py"]:::utility

    %% Visualization Tools
    subgraph "Visualization" 
        direction TB
        Vis1["visualize_output.py"]:::utility
        Vis2["visualize_output_directory_stage1.py"]:::utility
        Vis3["visualize_output_directory_stage2.py"]:::utility
    end

    %% Data Flow
    RawData -->|raw images & point clouds| GenPanoptic
    GenPanoptic --> SetupSplit
    SetupSplit --> SetupRadarIn
    SetupRadarIn --> SetupRadarTestIn
    SetupRadarTestIn --> SetupFinalTest
    SetupFinalTest --> DerivedData

    DerivedData --> RNTrainEntry
    RNTrainEntry --> RNTrainMain
    RNTrainMain --> RNCheckpoints
    RNCheckpoints --> FNTrainEntry
    DerivedData --> FNTrainEntry

    RNCheckpoints --> FNModel
    FNTrainEntry --> FNTrainMain
    FNTrainMain --> FNCheckpoints

    RNCheckpoints --> RNInferEntry
    RNInferEntry --> RNEval
    RNEval --> RNSave
    RNSave --> RNSaveUtils

    FNCheckpoints --> FNInferEntry
    FNInferEntry --> FNEvalUtils
    FNEvalUtils --> FNSave

    RNTrainMain -->|logs| LogUtils
    FNTrainMain -->|logs| LogUtils
    RNEval -->|logs| LogUtils
    FNEvalUtils -->|logs| LogUtils

    FNSave --> Vis1
    FNSave --> Vis2
    FNSave --> Vis3

    %% Bash connections
    BashSetup1 --> GenPanoptic
    BashSetup2 --> SetupRadarIn
    BashSetup3 --> SetupRadarTestIn
    BashSetup4 --> SetupFinalTest

    BashTrainRN --> RNTrainEntry
    BashRunRN --> RNInferEntry

    BashTrainFN --> FNTrainEntry
    BashRunFN --> FNInferEntry
    BashRunFNTest --> FNInferEntry

    %% Click Events
    click GenPanoptic "https://github.com/nesl/radar-camera-fusion-depth/blob/main/setup/gen_panoptic_seg.py"
    click SetupSplit "https://github.com/nesl/radar-camera-fusion-depth/blob/main/setup/setup_dataset_nuscenes.py"
    click SetupRadarIn "https://github.com/nesl/radar-camera-fusion-depth/blob/main/setup/setup_dataset_nuscenes_radarnet.py"
    click SetupRadarTestIn "https://github.com/nesl/radar-camera-fusion-depth/blob/main/setup/setup_dataset_nuscenes_radarnet_test.py"
    click SetupFinalTest "https://github.com/nesl/radar-camera-fusion-depth/blob/main/setup/setup_dataset_nuscenes_test.py"
    click BashSetup1 "https://github.com/nesl/radar-camera-fusion-depth/blob/main/bash/setup_dataset_nuscenes.sh"
    click BashSetup2 "https://github.com/nesl/radar-camera-fusion-depth/blob/main/bash/setup_dataset_nuscenes_radarnet.sh"
    click BashSetup3 "https://github.com/nesl/radar-camera-fusion-depth/blob/main/bash/setup_dataset_nuscenes_radarnet_test.sh"
    click BashSetup4 "https://github.com/nesl/radar-camera-fusion-depth/blob/main/bash/setup_dataset_nuscenes_test.sh"
    click RNModel "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/radarnet_model.py"
    click RNTransforms "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/radarnet_transforms.py"
    click RNTrainEntry "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/train_radarnet.py"
    click RNTrainMain "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/radarnet_main.py"
    click BashTrainRN "https://github.com/nesl/radar-camera-fusion-depth/blob/main/bash/train_radarnet_nuscenes.sh"
    click BashRunRN "https://github.com/nesl/radar-camera-fusion-depth/blob/main/bash/run_radarnet_nuscenes.sh"
    click RNInferEntry "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/run_radarnet.py"
    click RNEval "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/eval_stage_1_depth.py"
    click RNSave "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/save_stage_1_depth.py"
    click RNSaveUtils "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/save_stage_1_utils.py"
    click FNModel "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/fusionnet_model.py"
    click FNLosses "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/fusionnet_losses.py"
    click FNTransforms "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/fusionnet_transforms.py"
    click FNTrainEntry "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/train_fusionnet.py"
    click FNTrainMain "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/fusionnet_main.py"
    click BashTrainFN "https://github.com/nesl/radar-camera-fusion-depth/blob/main/bash/train_fusionnet_nuscenes.sh"
    click BashRunFN "https://github.com/nesl/radar-camera-fusion-depth/blob/main/bash/run_fusionnet_nuscenes.sh"
    click BashRunFNTest "https://github.com/nesl/radar-camera-fusion-depth/blob/main/bash/run_fusionnet_nuscenes_test.sh"
    click FNInferEntry "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/run_fusionnet.py"
    click FNEvalUtils "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/eval_utils.py"
    click FNSave "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/save_depth_radar.py"
    click LogUtils "https://github.com/nesl/radar-camera-fusion-depth/blob/main/src/log_utils.py"
    click Vis1 "https://github.com/nesl/radar-camera-fusion-depth/blob/main/tools/visualize_output.py"
    click Vis2 "https://github.com/nesl/radar-camera-fusion-depth/blob/main/tools/visualize_output_directory_stage1.py"
    click Vis3 "https://github.com/nesl/radar-camera-fusion-depth/blob/main/tools/visualize_output_directory_stage2.py"

    %% Styles
    classDef external fill:#b3cde0,stroke:#03396c,color:#03396c;
    classDef dataStore fill:#ccebc5,stroke:#238b45,color:#238b45;
    classDef script fill:#fed9a6,stroke:#e6550d,color:#e6550d;
    classDef model fill:#decbe4,stroke:#7b3294,color:#7b3294;
    classDef bash fill:#ffffcc,stroke:#b15928,color:#b15928;
    classDef utility fill:#fbb4ae,stroke:#b30000,color:#b30000;
```
