# zju3dv/street_gaussians architecture

> Architecture diagram of the GitHub repository zju3dv/street_gaussians, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zju3dv/street_gaussians
- Repository: https://github.com/zju3dv/street_gaussians

## Overview

##### Main Components
- Configuration Manager – This includes YAML configuration files in the “configs” folder and the Python modules found under lib/config. They define the parameters for training, rendering, and evaluation.
- Models and Rendering – The “lib/models” folder contains numerous components: different Gaussian models (e.g., actor, background, sky, street variants), camera pose handling, neural network GUI, and differentiable renderers. These pieces are interconnected, working together for scene synthesis and rendering.
- GPU-accelerated Submodules – The project uses submodules for performance-critical tasks:
– Diffusion Gaussian Rasterization is implemented in the “submodules/diff-gaussian-rasterization” directory. This portion includes both CUDA (.cu) files and CPU wrappers.
– Simple KNN – located under “submodules/simple-knn” and providing spatial querying and neighbor search functionality.
– Also, there is a submodule for the Waymo open-dataset reader for data extraction.
- Training and Rendering Scripts – Outside the modules, the user-facing scripts (e.g., train.py, render.py, make_ply.py) and shell scripts in “script/waymo” orchestrate the execution flow. They load configurations, invoke data pipelines, execute training loops, and trigger rendering.
- Visualization – After training and rendering, the system outputs formats (for instance, via make_ply.py) that can be viewed using external viewers (e.g., SIBR_viewers).

##### Relationships and Data Flow
- Data ingestion: Raw input (e.g., Waymo dataset, tracking predictions) is processed by conversion scripts. The dataset modules read this processed data and feed batches to the training modules.
- Training: The models (defined in lib/models) use processed dataset information along with auxiliary utilities (e.g., for loss calculation, camera geometry, etc., found in lib/utils) and GPU-accelerated operations from the diff-gaussian-rasterization submodule.
- Rendering: Once trained, the rendering pipeline uses a similar configuration path (render.py and accompanying scripts) to synthesize images from learned representations.
- Visualization: The output from rendering/training flows into post-processing (e.g., make_ply.py) for visualization and external viewer compatibility.

##### Important Architectural Patterns and Design Principles
- Modular Separation – Components are separated by function (data preprocessing, model definition, rendering, and utility libraries) making it easier to isolate and test modules.
- Pipeline Architecture – The system is built around sequential stages (data conversion → training → rendering → visualization) with scripts orchestrating transitions. This can be depicted as a linear pipeline with branching for sub-processes (e.g., multiple ways to preprocess or render).
- Hardware Acceleration – The use of dedicated CUDA-based submodules emphasizes the need to distinguish between CPU-bound logic and GPU-accelerated computation.
- Reusability and Extensibility – The clear separation into lib/ (core modules), script/ (entry point scripts), and submodules allows integration with external tools and reuse of components.

##### Technologies, Frameworks, and Libraries
- Python and PyTorch for model training and scripting.
- CUDA (C++/CUDA files in diff-gaussian-rasterization) for GPU-accelerated rendering.
- YAML for configuration.
- External libraries for image, graphics, and data utilities (as seen in lib/utils).
- Optional integration with external tools (SIBR_viewers) for visualization.
- Additional dependencies from the “requirements.txt” and installation instructions.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Manager Cluster
    subgraph "Configuration Manager"
        YAMLConfig["YAML Config Files"] 
        ConfigModules["Python Config Modules"]
    end

    %% Dataset and Preprocessing Pipeline Cluster
    subgraph "Dataset and Preprocessing Pipeline"
        DatasetModules["Dataset Modules"]
        WaymoScripts["Conversion Scripts (Waymo)"]
    end

    %% Training & Rendering Orchestrator Cluster
    subgraph "Training & Rendering Orchestrator"
        TrainScript["Training Script (train.py)"]
        RenderScript["Rendering Script (render.py)"]
        ShellScripts["Orchestration Scripts"]
    end

    %% Models and Differentiable Renderer Cluster
    subgraph "Models and Differentiable Renderer"
        GaussianModels["Gaussian Models & Renderer"]
        Visualizers["Visualization Tools"]
    end

    %% GPU Accelerated Operations Cluster
    subgraph "GPU Accelerated Operations"
        DiffRaster["Diffusion Gaussian Rasterization"]
        SimpleKNN["Simple KNN"]
        WaymoReader["Waymo Open-dataset Reader"]
    end

    %% Visualization & Export Cluster
    subgraph "Visualization & Export"
        PLYExport["PLY Export (make_ply.py)"]
    end

    %% Data Flow Connections
    %% Configuration Manager to Training & Rendering
    YAMLConfig -->|"config"| TrainScript
    ConfigModules -->|"config"| RenderScript

    %% Dataset Pipeline to Training & Rendering
    DatasetModules -->|"rawData"| TrainScript
    WaymoScripts -->|"convertData"| TrainScript

    %% Orchestration within Training & Rendering
    ShellScripts -->|"orchestrate"| TrainScript
    ShellScripts -->|"orchestrate"| RenderScript

    %% Training triggering Model operations
    TrainScript -->|"trainModels"| GaussianModels
    RenderScript -->|"renderCall"| GaussianModels

    %% GPU Accelerated Operations involvement
    TrainScript -->|"GPUOp"| DiffRaster
    TrainScript -->|"GPUOp"| SimpleKNN
    TrainScript -->|"GPUOp"| WaymoReader
    RenderScript -->|"GPUOp"| DiffRaster

    DiffRaster -->|"accelerates"| GaussianModels
    SimpleKNN -->|"assist"| GaussianModels
    WaymoReader -->|"assist"| GaussianModels

    %% Models and Rendering to Visualization
    GaussianModels -->|"output"| PLYExport
    GaussianModels -->|"visualize"| Visualizers

    %% Styles
    classDef config fill:#f9e6ff,stroke:#800080,stroke-width:2px;
    classDef pipeline fill:#e6f7ff,stroke:#0066cc,stroke-width:2px;
    classDef orchestrator fill:#e6ffe6,stroke:#008000,stroke-width:2px;
    classDef models fill:#fff0e6,stroke:#ff6600,stroke-width:2px;
    classDef gpu fill:#fff9e6,stroke:#cc9900,stroke-width:2px;
    classDef viz fill:#ffe6e6,stroke:#cc0000,stroke-width:2px;

    class YAMLConfig,ConfigModules config;
    class DatasetModules,WaymoScripts pipeline;
    class TrainScript,RenderScript,ShellScripts orchestrator;
    class GaussianModels,Visualizers models;
    class DiffRaster,SimpleKNN,WaymoReader gpu;
    class PLYExport viz;

    %% Click Events for Component Mapping
    click YAMLConfig "https://github.com/zju3dv/street_gaussians/tree/main/configs"
    click ConfigModules "https://github.com/zju3dv/street_gaussians/tree/main/lib/config"
    click DatasetModules "https://github.com/zju3dv/street_gaussians/tree/main/lib/datasets"
    click WaymoScripts "https://github.com/zju3dv/street_gaussians/tree/main/script/waymo"
    click GaussianModels "https://github.com/zju3dv/street_gaussians/tree/main/lib/models"
    click Visualizers "https://github.com/zju3dv/street_gaussians/tree/main/lib/visualizers"
    click DiffRaster "https://github.com/zju3dv/street_gaussians/tree/main/submodules/diff-gaussian-rasterization"
    click SimpleKNN "https://github.com/zju3dv/street_gaussians/tree/main/submodules/simple-knn"
    click WaymoReader "https://github.com/zju3dv/street_gaussians/tree/main/submodules/simple-waymo-open-dataset-reader"
    click TrainScript "https://github.com/zju3dv/street_gaussians/blob/main/train.py"
    click RenderScript "https://github.com/zju3dv/street_gaussians/blob/main/render.py"
    click ShellScripts "https://github.com/zju3dv/street_gaussians/tree/main/script"
    click PLYExport "https://github.com/zju3dv/street_gaussians/blob/main/make_ply.py"
```
