# hailanyi/virconv architecture

> Architecture diagram of the GitHub repository hailanyi/virconv, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hailanyi/virconv
- Repository: https://github.com/hailanyi/virconv

## Overview

##### Project Type and Purpose
- This isn’t a traditional full-stack application; it is a research/detection framework for 3D object detection in autonomous driving (using the KITTI dataset) with both supervised and semi-supervised variants.
- Its purpose is to provide multiple detection frameworks (VirConv-L, VirConv-T, VirConv-S) that incorporate virtual sparse convolution and refined detection pipelines.

##### File Structure and Its Implications
- Top-Level Directories:
– The “data” folder holds the KITTI dataset with various splits (training, testing, semi) and additional resources like ImageSets.
– The “pcdet” directory is the core codebase for the detection pipeline. It includes modules for datasets, model components (backbones, dense heads, ROI heads), processors, and utility functions.
– The “tools” directory contains scripts for training, testing, data creation, visualization, and configuration files (yaml) that detail experiment parameters.
- Inside “pcdet” you have:
– datasets: Contains dataset core classes, augmentation modules (e.g., virtual point augmentation, data augmentor utilities), and dataset loaders for KITTI.
– models: Divided into several internal subdirectories for different components (backbones 2D/3D, dense heads, ROI heads, target assigners) that form the main detection network.
– ops: Multiple subdirectories that house custom operations typically implemented in C++/CUDA, encapsulating Spconv operations, PointNet2 operations, ROI aware pooling, etc.
- “tools” includes scripts (e.g., train.py, test.py), configuration files for various models, and support for the PENet-based depth completion module.

A. Major Components to Include:
- Dataset & Data Preparation:
– Raw KITTI datasets (training, testing, odometry for the semi dataset).
– Data augmentation and preprocessing modules (located under pcdet/datasets/augmentor & processor).
– Separate pipelines for generating “semi” or “velodyne_depth” versions.
- Model Architecture:
– Backbone Networks: Illustrate separate sub-modules for 2D (e.g., BEV backbones) and 3D (e.g., PointNet2, spconv_backbone, spconv_unet) processing.
– Feature Encoding: Include components like “point_feature_encoder” and “voxel_set_abstraction” under the pcdet models.
– Detection Heads: Show dense heads (anchor_head, center_head) and ROI heads (roi_head_template, target assigner modules).
– Model Utility Modules: Display “model_utils” and “loss_utils”, which support the detection components.
- Custom Ops Layer:
– Mark all the custom CUDA/C++ kernels under pcdet/ops (dcn, iou3d_nms, pointnet2, roiaware_pool3d, roipoint_pool3d, votr_ops) that provide high-performance operations.
- Training & Evaluation Pipeline:
– Training scripts, distributed training (e.g., dist_train.sh), and evaluation (test.py, dist_test.sh) are all encapsulated in the “tools” directory.
– Configuration management using YAML configuration files (tools/cfgs) should be clearly noted as they define experiment parameters.
- Dependency & External Modules:
– External dependencies (PyTorch, Spconv, Numba, CUDA) should be noted and linked to the ops and overall model computations.
– The integration with modules like PENet for depth completion must also be mentioned.

B. Relationships and Data Flow Among Components:
- Data then flows into the “point_feature_encoder” which converts raw point cloud data into structured features.
- These features are then passed to distinct backbone modules (2D and 3D) that extract higher-level representations.
- The extracted features flow into Detection Heads (dense, anchor or center head, and ROI head for refinement), resulting in predicted objects.
- Custom Ops (CUDA kernels, specialized pooling, and attention operations) are called throughout the processing pipeline to ensure efficient computation.
- The training and evaluation scripts (in “tools”) control the overall orchestration, reading configurations, initializing models with these components, and managing data flow during training/inference.
- Feedback loops exist in the training pipeline (loss utilities feed error metrics back to update the model weights).

C. Architectural Patterns and Design Principles:
- Layered Architecture: Indicate how lower-level operations (custom ops, feature encoding) underpin higher-level tasks (detection and refinement).
- Configurability: Emphasize that configuration files (YAML) drive the instantiation of these modules, which means the architecture is highly configurable.
- Parallelism: Note that the project supports multi-GPU training (distributed training scripts reflect this).

D. Relevant Technologies and Frameworks:
- PyTorch is the core deep learning framework.
- Spconv is heavily used for sparse convolution operations.
- Custom CUDA kernels are used (under pcdet/ops).
- YAML configuration files manage model and dataset parameters.
- Dependencies such as Numba, Tensorboard, Open3D, and others support the training, debugging, and visualization tasks.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Preparation Subgraph
    subgraph "Data Preparation"
        RD["Raw Data (KITTI Dataset)"]:::dataset
        DA["Data Augmentor"]:::dataprep
        DP["Data Processor"]:::dataprep
        PFE["Point Feature Encoder"]:::dataprep
        FE["Feature Encoding & Voxelization"]:::feature
    end

    %% Model Architecture Subgraph
    subgraph "Model Architecture"
        B2D["2D Backbone"]:::backbone
        B3D["3D Backbone"]:::backbone
        subgraph "Detection Heads"
            DH1["Dense Heads"]:::detector
            RH["ROI Heads"]:::detector
        end
    end

    %% Custom Ops Subgraph
    subgraph "Custom Ops"
        COP["Custom Ops"]:::ops
    end

    %% Utility Modules Subgraph
    subgraph "Utility Modules"
        MU["Model Utilities"]:::utils
        UT["Utils"]:::utils
    end

    %% Training & Evaluation Pipeline Subgraph
    subgraph "Training & Evaluation"
        TE["Training & Evaluation Controller"]:::training
        TScript["Training Script"]:::tools
        TTest["Testing Script"]:::tools
        CFG["Configuration Files"]:::tools
        EVAL["Evaluation Utilities"]:::tools
        TRAINUTIL["Training Utilities"]:::tools
        PENET["PENet Module"]:::tools
    end

    %% Data Preparation Flow
    RD -->|"feeds"| DA
    DA -->|"processes"| DP
    DP -->|"formats"| PFE
    PFE -->|"encodes"| FE

    %% Model Architecture Flow
    FE -->|"input"| B2D
    FE -->|"input"| B3D
    B2D -->|"features"| DH1
    B3D -->|"features"| RH

    %% Custom Ops Influence Flow
    COP ---|"accelerates"| B3D
    COP ---|"enhances"| RH

    %% Integration with Training & Evaluation
    DH1 -->|"detection outputs"| TE
    RH -->|"detection outputs"| TE
    MU -->|"supports"| TE
    UT -->|"supports"| TE

    %% Training & Evaluation Detailed Flow
    TE -->|"triggers"| TScript
    TE -->|"triggers"| TTest
    TE -->|"driven_by"| CFG
    TE -->|"evaluates_with"| EVAL
    TE -->|"optimizes_with"| TRAINUTIL
    TE -->|"integrates"| PENET

    %% Styling Classes
    classDef dataset fill:#ffebcd,stroke:#8b4513,stroke-width:2px;
    classDef dataprep fill:#e0ffff,stroke:#008b8b,stroke-width:2px;
    classDef feature fill:#ffe4e1,stroke:#cd5c5c,stroke-width:2px;
    classDef backbone fill:#e6e6fa,stroke:#9370db,stroke-width:2px;
    classDef detector fill:#f0fff0,stroke:#228b22,stroke-width:2px;
    classDef ops fill:#f5f5dc,stroke:#daa520,stroke-width:2px;
    classDef training fill:#f0f8ff,stroke:#4682b4,stroke-width:2px;
    classDef tools fill:#fafad2,stroke:#b8860b,stroke-width:2px;
    classDef utils fill:#ffe4b5,stroke:#cd853f,stroke-width:2px;

    %% Click Events
    click RD "https://github.com/hailanyi/virconv/tree/master/data/kitti"
    click DA "https://github.com/hailanyi/virconv/tree/master/pcdet/datasets/augmentor"
    click DP "https://github.com/hailanyi/virconv/tree/master/pcdet/datasets/processor"
    click PFE "https://github.com/hailanyi/virconv/blob/master/pcdet/datasets/processor/point_feature_encoder.py"
    click FE "https://github.com/hailanyi/virconv/blob/master/pcdet/models/backbones_3d/pfe/voxel_set_abstraction.py"
    click B2D "https://github.com/hailanyi/virconv/tree/master/pcdet/models/backbones_2d"
    click B3D "https://github.com/hailanyi/virconv/tree/master/pcdet/models/backbones_3d"
    click DH1 "https://github.com/hailanyi/virconv/tree/master/pcdet/models/dense_heads"
    click RH "https://github.com/hailanyi/virconv/tree/master/pcdet/models/roi_heads"
    click COP "https://github.com/hailanyi/virconv/tree/master/pcdet/ops"
    click MU "https://github.com/hailanyi/virconv/tree/master/pcdet/models/model_utils"
    click UT "https://github.com/hailanyi/virconv/tree/master/pcdet/utils"
    click TScript "https://github.com/hailanyi/virconv/blob/master/tools/train.py"
    click TTest "https://github.com/hailanyi/virconv/blob/master/tools/test.py"
    click CFG "https://github.com/hailanyi/virconv/tree/master/tools/cfgs"
    click EVAL "https://github.com/hailanyi/virconv/blob/master/tools/eval_utils/eval_utils.py"
    click TRAINUTIL "https://github.com/hailanyi/virconv/tree/master/tools/train_utils"
    click PENET "https://github.com/hailanyi/virconv/tree/master/tools/PENet"
```
