# opendrivelab/occnet architecture

> Architecture diagram of the GitHub repository opendrivelab/occnet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opendrivelab/occnet
- Repository: https://github.com/opendrivelab/occnet

## Overview

##### Main Components
- Data Sources and Input: Represent the multiple input modalities (e.g., cameras from the nuScenes dataset), and show how raw images and sensor meta information are ingested.
- Dataset and Configuration Management: Include components for handling the occupancy and flow dataset as well as the configuration folders (projects/configs and its subfolders holding dataset, model, and schedule configurations).
- Backbone networks (see projects/mmdet3d_plugin/models/backbones)
- Detection and occupancy heads (from projects/mmdet3d_plugin/models and the bevformer-specific detectors and dense_heads in the mmdet3d_plugin/bevformer folder)
- Transformer layers and attention modules (in projects/mmdet3d_plugin/bevformer/modules)
- Training and Inference Pipelines:
- The training scripts (tools/train.py, tools/dist_train.sh) and runner modules (projects/mmdet3d_plugin/bevformer/runner) should be shown.
- The inference pipeline and evaluation (tools/dist_test.sh, projects/mmdet3d_plugin/datasets for data pipelines).
- Evaluation and Metrics:
- A separate evaluation module for computing ray-based mIoU and occupancy flow metrics (projects/mmdet3d_plugin/datasets/ray_metrics.py, tools/ray_iou).
- Submission Conversion:
- Process of converting output predictions into the standardized submission format, detailing the scripts in tools/ray_iou and their integration with the rest of the pipeline.
- Utilities:
- Mention auxiliary scripts (like utils/vis.py for visualization, docs for getting started) and external dependencies (such as BEVFormer) that support the system.

##### Relationships and Interactions
- Data flow starting from raw input (camera images and nuScenes meta files) through preprocessing pipelines (projects/mmdet3d_plugin/datasets/pipelines) to the network’s forward pass.
- Bidirectional relationship between training/inference scripts (tools/ folder) and the model/plugins, highlighting how training scripts call the runner modules which in turn invoke model components.
- Evaluation module’s connection to both prediction outputs and ground-truth data, indicating the use of ray projection (tools/ray_iou/ray_casting.py) and metric aggregation.
- Submission module that packages final predictions into the required submission structure (as detailed in the README).

##### Architectural Patterns and Design Principles
- Modular Design: Emphasize that the project is split into modules by functionality (e.g., configurations, models, datasets, utilities) for flexibility and extensibility.
- Plugin Architecture: Show that custom plugins extend MMDetection3D’s capabilities (located in projects/mmdet3d_plugin) allowing new model heads, attention mechanisms, and evaluation routines.
- Separation of Concerns: Clearly separate training, inference, evaluation, and submission preparation into distinct pipelines.
- Config-driven Design: Stress how the base and specialized configuration files are used to parameterize and control the model’s behavior without hardcoding details.

- Clear labels for each component (e.g., “Image Input”, “Configuration Manager”, “Data Pipeline”, “Backbone Network”, “Transformer Encoder/Decoder”, “Detection Heads”, “Evaluation Module”, “Submission Formatter”).
- Processing modules (rectangles)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Preprocessing"
        DS["Image & Sensor Data (nuScenes)"]:::datasource
        DP["Data Processing Pipeline (pipelines & loaders)"]:::pipeline
    end
    DS --> DP

    subgraph "Configuration"
        CF["Configuration Manager (configs, _base_, bevformer)"]:::config
    end
    DP --> CF

    subgraph "Core Model"
        BN["Backbone Network"]:::core
        DH["Occupancy Dense Heads"]:::core
        DET["Detection Heads"]:::core
        TA["Transformer & Attention Modules"]:::core
    end
    CF --> BN
    CF --> DH
    CF --> DET
    CF --> TA

    subgraph "Training & Inference"
        TP["Training Pipeline (train.py, dist_train.sh)"]:::pipeline
        RU["Runner Modules"]:::pipeline
        IP["Inference Pipeline (dist_test.sh)"]:::pipeline
    end
    BN --> TP
    DH --> TP
    DET --> TP
    TA --> TP
    TP --> RU
    RU --> IP

    subgraph "Evaluation & Submission"
        EV["Evaluation & Metrics (ray_metrics.py)"]:::eval
        SF["Submission Formatter (ray_iou)"]:::eval
    end
    IP --> EV
    EV --> SF

    subgraph "Utilities"
        UT["Visualization & Utilities (vis.py, docs)"]:::util
    end
    UT --> DP

    %% Click Events
    click DS "https://github.com/opendrivelab/occnet/tree/challenge/projects/mmdet3d_plugin/datasets"
    click CF "https://github.com/opendrivelab/occnet/tree/challenge/projects/configs"
    click BN "https://github.com/opendrivelab/occnet/tree/challenge/projects/mmdet3d_plugin/models/backbones"
    click DH "https://github.com/opendrivelab/occnet/tree/challenge/projects/mmdet3d_plugin/bevformer/dense_heads"
    click DET "https://github.com/opendrivelab/occnet/tree/challenge/projects/mmdet3d_plugin/bevformer/detectors"
    click TA "https://github.com/opendrivelab/occnet/tree/challenge/projects/mmdet3d_plugin/bevformer/modules"
    click TP "https://github.com/opendrivelab/occnet/blob/challenge/tools/train.py"
    click RU "https://github.com/opendrivelab/occnet/tree/challenge/projects/mmdet3d_plugin/bevformer/runner"
    click IP "https://github.com/opendrivelab/occnet/blob/challenge/tools/dist_test.sh"
    click EV "https://github.com/opendrivelab/occnet/blob/challenge/projects/mmdet3d_plugin/datasets/ray_metrics.py"
    click SF "https://github.com/opendrivelab/occnet/tree/challenge/tools/ray_iou"
    click UT "https://github.com/opendrivelab/occnet/blob/challenge/utils/vis.py"

    %% Styles
    classDef datasource fill:#ffcccc,stroke:#b30000,stroke-width:2px;
    classDef pipeline fill:#ffffcc,stroke:#b3b300,stroke-width:2px;
    classDef config fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef core fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef eval fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef util fill:#e2e3e5,stroke:#383d41,stroke-width:2px;
```
