# hbb1/2d-gaussian-splatting architecture

> Architecture diagram of the GitHub repository hbb1/2d-gaussian-splatting, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hbb1/2d-gaussian-splatting
- Repository: https://github.com/hbb1/2d-gaussian-splatting

## Overview

##### Project Type and Purpose
- This is not a web app or microservice system but a research-oriented Python pipeline for 2D Gaussian Splatting.
- Its main goal is to ingest multi-view scene data (COLMAP or NeRF), train a differentiable renderer (surface elements represented as oriented 2D disks), extract meshes, evaluate quality, and provide both CLI and GUI visualization.

##### High-Level Component Breakdown
a. Data Ingestion & Preprocessing
- Directories:
- scene/ (colmap_loader.py, dataset_readers.py)
- arguments/ (parses CLI flags)
- Role: load images, camera intrinsics/extrinsics, prepare input structures.
b. Core Training & Rendering Engine
- Modules:
- gaussian_renderer/ (network_gui.py orchestrates forward/backward passes)
- train.py (entry point for training)
- render.py (mesh extraction via TSDF fusion)
- Dependencies:
- submodules/diff-surfel-rasterization (CUDA operator fusing)
- lpipsPyTorch/ (perceptual loss)
- PyTorch as primary DL framework
c. Losses & Metrics
- Files: metrics.py, utils/loss_utils.py
- Role: compute photometric, geometric regularizers (normal consistency, distortion), perceptual losses.
d. Mesh Extraction & TSDF Fusion
- render.py calls Open3D under the hood (via scene/gaussian_model.py) to do bounded/unbounded TSDF fusion.
e. Evaluation Suite
- scripts/ (dtu_eval.py, tnt_eval.py, nerf_eval.py, etc.)
- Each subfolder (eval_dtu, eval_tnt, etc.) contains dataset-specific pipelines: mesh registration, Chamfer/F1 metrics, plotting.
f. Visualization
- view.py (CLI for launching a lightweight viewer)
- gaussian_renderer/network_gui.py (PyQt or similar GUI for monitoring during training)
- External: optional remote/web viewers via submodules or partner repos.
g. Environment & Configuration
- environment.yml (conda spec)
- .gitmodules (pulls in diff-surfel-rasterization and simple-knn)
- scripts/eval_tnt/requirements.txt (pin Open3D version)

##### Relationships and Data Flow
1. User provides dataset path → arguments module parses flags → scene loaders produce camera/image tensors
2. train.py invokes gaussian_renderer:
- forward/render step uses diff-surfel-rasterization → produces rendered images
- backward step uses losses (metrics.py + lpipsPyTorch) → updates Gaussian parameters
- checkpoint saved to disk
3. render.py loads checkpoint + scene → extracts mesh via TSDF fusion (Open3D) → writes mesh file (PLY)
4. scripts/… evaluation scripts read rendered images or extracted mesh + ground truth → compute metrics → output tables/plots
5. view.py (or network_gui.py) loads model + dataset for interactive inspection

##### Architectural Patterns & Design Principles
- Modular package structure: scene, utils, metrics, renderer, scripts, submodules
- CLI-driven workflows: each major step (train, render, eval, view) is a separate script
- Separation of concerns:
- Data loaders vs. Core engine vs. Evaluation
- External heavy-compute kernels isolated in submodules
- Plugin-style external libs: lpips already wrapped, diff-surfel fused via submodule

##### Final Advice
- Keep labels concise but precise: use terms straight from the repo (e.g., “gaussian_renderer”, “render.py”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    Dataset(("Multi-view Dataset")):::data

    %% Ingestion & Preprocessing
    subgraph "Ingestion & Preprocessing"
        ArgsInit["arguments/__init__.py"]:::data
        ColmapLoader["scene/colmap_loader.py"]:::data
        DatasetReaders["scene/dataset_readers.py"]:::data
        Cameras["scene/cameras.py"]:::data
    end

    %% Core Training & Rendering Engine
    subgraph "Core Training & Rendering Engine"
        Train["train.py"]:::core
        Render["render.py"]:::core
        NetworkGUI["gaussian_renderer/network_gui.py"]:::core
        subgraph "Submodules"
            DiffSurfel["diff-surfel-rasterization"]:::external
            SimpleKNN["simple-knn"]:::external
        end
        PyTorch(("PyTorch")):::external
    end

    %% Losses & Metrics
    subgraph "Losses & Metrics"
        Metrics["metrics.py"]:::core
        LossUtils["utils/loss_utils.py"]:::core
        LPIPS["lpipsPyTorch/modules/lpips.py"]:::external
    end

    %% Mesh Extraction & TSDF Fusion
    subgraph "Mesh Extraction & TSDF Fusion"
        GaussianModel["scene/gaussian_model.py"]:::core
        Mesh(("Extracted Mesh (PLY)")):::data
    end

    %% Evaluation Suite
    subgraph "Evaluation Suite"
        DTUEval["scripts/dtu_eval.py"]:::eval
        TNTEval["scripts/tnt_eval.py"]:::eval
        NeRfEval["scripts/nerf_eval.py"]:::eval
        M360Eval["scripts/m360_eval.py"]:::eval
        Summary["scripts/summary.py"]:::eval
        subgraph "DTU Pipeline"
            EvalDTU["scripts/eval_dtu/"]:::eval
        end
        subgraph "TnT Pipeline"
            EvalTNT["scripts/eval_tnt/"]:::eval
            TNTReq["scripts/eval_tnt/requirements.txt"]:::external
        end
    end

    %% Visualization
    subgraph "Visualization"
        Viewer["view.py"]:::vis
        NetworkGUI
    end

    %% Environment & Configuration
    subgraph "Environment & Configuration"
        EnvYML["environment.yml"]:::external
        GitModules[".gitmodules"]:::external
    end

    %% Data Flow
    Dataset -->|"images + intrinsics"| ArgsInit
    ArgsInit --> ColmapLoader
    ColmapLoader --> DatasetReaders
    DatasetReaders --> Cameras
    Cameras --> Train
    Train -->|forward/backward| NetworkGUI
    NetworkGUI --> DiffSurfel
    NetworkGUI --> SimpleKNN
    Train -->|losses| Metrics
    Metrics --> LossUtils
    Metrics --> LPIPS
    Train -->|checkpoints| Mesh
    Mesh --> Render
    Render --> GaussianModel
    GaussianModel --> Mesh
    Mesh --> DTUEval
    Mesh --> TNTEval
    Mesh --> NeRfEval
    Mesh --> M360Eval
    DTUEval --> EvalDTU
    TNTEval --> EvalTNT
    TNTEval --> TNTReq
    NeRfEval --> Summary
    Mesh --> Viewer
    Mesh --> NetworkGUI
    EnvYML -.-> Train
    GitModules -.-> DiffSurfel
    GitModules -.-> SimpleKNN

    %% Click Events
    click ArgsInit "https://github.com/hbb1/2d-gaussian-splatting/blob/main/arguments/__init__.py"
    click ColmapLoader "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scene/colmap_loader.py"
    click DatasetReaders "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scene/dataset_readers.py"
    click Cameras "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scene/cameras.py"
    click Train "https://github.com/hbb1/2d-gaussian-splatting/blob/main/train.py"
    click Render "https://github.com/hbb1/2d-gaussian-splatting/blob/main/render.py"
    click NetworkGUI "https://github.com/hbb1/2d-gaussian-splatting/blob/main/gaussian_renderer/network_gui.py"
    click DiffSurfel "https://github.com/hbb1/2d-gaussian-splatting/tree/main/submodules/diff-surfel-rasterization"
    click SimpleKNN "https://github.com/hbb1/2d-gaussian-splatting/tree/main/submodules/simple-knn"
    click Metrics "https://github.com/hbb1/2d-gaussian-splatting/blob/main/metrics.py"
    click LossUtils "https://github.com/hbb1/2d-gaussian-splatting/blob/main/utils/loss_utils.py"
    click LPIPS "https://github.com/hbb1/2d-gaussian-splatting/blob/main/lpipsPyTorch/modules/lpips.py"
    click GaussianModel "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scene/gaussian_model.py"
    click DTUEval "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scripts/dtu_eval.py"
    click TNTEval "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scripts/tnt_eval.py"
    click NeRfEval "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scripts/nerf_eval.py"
    click M360Eval "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scripts/m360_eval.py"
    click Summary "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scripts/summary.py"
    click EvalDTU "https://github.com/hbb1/2d-gaussian-splatting/tree/main/scripts/eval_dtu/"
    click EvalTNT "https://github.com/hbb1/2d-gaussian-splatting/tree/main/scripts/eval_tnt/"
    click TNTReq "https://github.com/hbb1/2d-gaussian-splatting/blob/main/scripts/eval_tnt/requirements.txt"
    click Viewer "https://github.com/hbb1/2d-gaussian-splatting/blob/main/view.py"
    click EnvYML "https://github.com/hbb1/2d-gaussian-splatting/blob/main/environment.yml"
    click GitModules "https://github.com/hbb1/2d-gaussian-splatting/blob/main/.gitmodules"

    %% Styles
    classDef data fill:#e3fcef,stroke:#2d8f45,color:#000
    classDef core fill:#cce5ff,stroke:#005cbf,color:#000
    classDef eval fill:#ffe5b4,stroke:#cc8400,color:#000
    classDef vis fill:#f0ccff,stroke:#9c27b0,color:#000
    classDef external fill:#e2e3e5,stroke:#6c757d,color:#000
```
