# city-super/scaffold-gs architecture

> Architecture diagram of the GitHub repository city-super/scaffold-gs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/city-super/scaffold-gs
- Repository: https://github.com/city-super/scaffold-gs

## Overview

##### Major architectural layers and modules
- Data Ingestion & Preprocessing:
– Raw input data (images, COLMAP outputs, SfM point-cloud data) organized under data/ – see the “Data” section in the README.
– Preprocessing tools (scripts in “dataset_tools” and various converters in SIBR_viewers’s documentation for Colmap, etc.).
- Core Training Pipeline:
– The Python training scripts (e.g. train.py, train_xxx.sh, single_train.sh) initiate the learning process.
– Submodules like “diff-gaussian-rasterization” and “simple-knn” (in the submodules folder) indicate that CUDA-level rasterization and spatial data handling are used to accelerate rendering.
- Model Checkpointing & Configuration:
– There are configuration files and arguments (cfg_args) that control training (and even a system to update anchor points via voxel_size and update_init_factor parameters).
- Rendering & Evaluation Pipeline:
– The training code automatically triggers evaluation procedures: rendering results (using the integrated diff‐Gaussian rendering), fps estimation and quality metrics printed/logged.
– In addition, utilities such as render.py and metrics.py provide manual rendering and metric computations.
– Under “gaussian_renderer” and “scene” folders, additional tools for setting up the rendering scene exist.
- Interactive Viewer / SIBR_viewers:
– The “SIBR_viewers” directory contains the C++/CMake code, assets, and configuration to build an interactive viewer. This component is used for visualizing the final trained model thereby providing real‐time inspection.

##### Relationships and interactions
- Data Flow: Raw input data (images and SfM results) are preprocessed and fed into the training module. The training module uses these inputs to build the scene representation using an explicit anchor-based system (structured 3D Gaussians), learning per-anchor attributes with various MLP components.
- Between Training and Rendering: Once the training is complete, the checkpoint (point cloud and network weights) is used by the rendering pipeline (both automated evaluation in training code and the interactive viewer component).
- The rendering pipeline makes calls to low-level CUDA modules (diff-gaussian-rasterization) for fast rasterization and rendering computations.
- Metrics and quality evaluation are executed after rendering to compute metrics (e.g., error, fps) – these outcomes are logged.

##### Technical ecosystem and design patterns
- Layer Separation: Clearly separate data ingestion/preprocessing, training (and model optimization), rendering, and viewing.
- Heterogeneous implementation: The project is hybrid – a mix of C++/CUDA components (for performance rendering, viewer, and some core functionalities in SIBR_viewers) and Python modules (for training, evaluation, and utility scripts). Represent how the C++ and Python components interact (through saved model checkpoints, file I/O, and configuration files).
- External dependencies: Incorporate external libraries like Colmap (for SfM processing), LPIPS PyTorch (for perceptual loss metrics), and the third-party diff-gaussian-rasterization module.

1. Data Input & Preprocessing: “Input Data (Images, SfM/Colmap Outputs)”, “Dataset Tools & Converters”
3. Model Storage: “Checkpoint Files (point_cloud.ply, network weights .pt, cfg_args, cameras.json)”
4. Rendering & Evaluation: “Rendering Engine (integrating diff-gaussian-rasterization, evaluation modules: render.py, metrics.py)”
5. Viewing: “Interactive Viewer (SIBR_viewers – C++/CMake based interactive display)”
- Annotations like “CUDA acceleration”, “MLP-based attribute prediction”, and “External Library Dependencies (e.g., Colmap, LPIPS)”.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data & Preprocessing"
        D1["Input Data (Images, SfM/Colmap Outputs)"]
        D2["Dataset Tools & Converters"]
    end

    subgraph "Training Pipeline"
        T1_1["train.py"]
        T1_2["train.sh"]
        T2["Core Modules (assets, scene, system, video, view)"]
    end

    subgraph "Model Storage"
        M1["Checkpoint Files (.ply, .pt, cfg_args, cameras.json)"]
    end

    subgraph "Rendering & Evaluation"
        R1["Rendering Engine (render.py, metrics.py, gaussian_renderer, scene, renderer)"]
        R2["CUDA Rasterization (diff-gaussian)"]
        R3["Spatial Data Handling (Simple-KNN)"]
    end

    subgraph "Interactive Viewer"
        V1["Interactive Viewer (C++/CMake)"]
    end

    U1["Utility Scripts (camera_utils, graphics_utils, etc)"]

    %% Data Flow Connections
    D1 --> D2
    D2 --> T1_1
    D2 --> T1_2
    T1_1 --> T2
    T1_2 --> T2
    T2 --> M1
    M1 --> R1
    M1 --> V1
    R1 -->|"CUDA_acceleration"| R2
    R1 -->|"spatial_lookup"| R3
    U1 --> T2
    U1 --> R1
    V1 -->|"feedback"| T2

    %% Click Events
    click D2 "https://github.com/city-super/scaffold-gs/tree/main/SIBR_viewers/src/projects/dataset_tools"
    click T1_1 "https://github.com/city-super/scaffold-gs/blob/main/train.py"
    click T1_2 "https://github.com/city-super/scaffold-gs/blob/main/train.sh"
    click T2 "https://github.com/city-super/scaffold-gs/tree/main/SIBR_viewers/src/core"
    click R1 "https://github.com/city-super/scaffold-gs/blob/main/render.py"
    click R2 "https://github.com/city-super/scaffold-gs/tree/main/submodules/diff-gaussian-rasterization"
    click R3 "https://github.com/city-super/scaffold-gs/tree/main/submodules/simple-knn"
    click V1 "https://github.com/city-super/scaffold-gs/tree/main/SIBR_viewers/"
    click U1 "https://github.com/city-super/scaffold-gs/tree/main/utils/"

    %% Styles
    classDef data fill:#ABEBC6,stroke:#27AE60,stroke-width:2px;
    classDef python fill:#AED6F1,stroke:#2E86C1,stroke-width:2px;
    classDef cuda fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
    classDef cpp fill:#F9E79F,stroke:#D4AC0D,stroke-width:2px;
    classDef utility fill:#D7BDE2,stroke:#8E44AD,stroke-width:2px;

    class D1,D2,M1 data;
    class T1_1,T1_2,T2,R1 python;
    class R2,R3 cuda;
    class V1 cpp;
    class U1 utility;
```
