# humansensinglab/taming-3dgs architecture

> Architecture diagram of the GitHub repository humansensinglab/taming-3dgs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/humansensinglab/taming-3dgs
- Repository: https://github.com/humansensinglab/taming-3dgs

## Overview

##### Main System Components
- Data and Scene Management:
– The “scene” directory includes cameras.py, colmap_loader.py, dataset_readers.py, and gaussian_model.py. This module is responsible for loading the scene, camera parameters (from SfM/colmap), and building the gaussian-based scene representations.
- Training and Optimization:
– The top-level “train.py” (and its accompanying “train.sh”) drive the training procedure.
– The “arguments” package likely contains command–line parsing and configuration management.
– Modules like “utils” (which include camera_utils, loss_utils, graphics_utils, taming_utils, etc.) support various mathematical, imaging, and system functions.
– “lpipsPyTorch” is used to compute perceptual similarity metrics (LPIPS) for loss calculations, providing guidance for the optimization process.
- Rendering and Visualization:
– The “gaussian_renderer” folder contains modules for network GUI (both standard and websockets based) and supports interactive debugging or quick previews.
– The “web_viewer” directory (with app.js and render.html) implements a browser–based viewer that interacts via websockets (if enabled) to track training progress visually.
– “render.py” and possibly “full_eval.py” provide scripts for final image generation and evaluation.
- Low–Level Performance Optimizations:
– The submodule “diff-gaussian-rasterization” is a collection of C++ and CUDA implementations that handle differentiable rasterization, optimized to accelerate forward/backward passes. It integrates a third–party math library (GLM) and a custom CUDA kernel design.
- Evaluation and Metrics:
– “metrics.py” and code in lpipsPyTorch support system evaluation and enable reproducibility of report scores.

##### Component Relationships and Data Flow
– The training pipeline begins in “train.py”, where the input scene is read through modules in “scene”. The initial gaussian parameters come from SfM and then densification and optimization (using loss functions, LPIPS metrics, etc.) are performed.
– Throughout the training process, “utils” provide support functions for imaging and computational tasks.
– The diff-gaussian-rasterization submodule acts as a heavy–weight, optimized rendering engine that converts gaussian primitives into output images; it communicates with the higher–level training code.
– For visualization, data (e.g., intermediate rendered frames or network updates) are sent either directly to the “gaussian_renderer” or via websockets to the “web_viewer” for a browser-based display.

##### Architectural Patterns and Technologies
– Modular design: Each functional area (data loading, training, rendering, evaluation) is encapsulated in its own directory.
– Hybrid programming model: Python scripts and modules coordinate high–level processing while performance-critical parts are implemented in C++/CUDA under diff-gaussian-rasterization.
– Websockets and browser–based UI: This design supports remote monitoring of long training processes via a simple web server integrated into the training script.
– Use of established libraries: LPIPS from lpipsPyTorch for perceptual metrics, and GLM (via diff-gaussian-rasterization/third_party) for mathematics, reinforce a solid foundation.

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% Data and Scene Management
    subgraph "Data and Scene Management"
        Scene["Scene Module"]:::python
    end

    %% Training and Optimization
    subgraph "Training and Optimization"
        T1["train.py"]:::python
        TS["train.sh"]:::bash
        TArg["arguments"]:::python
        TUtil["utils"]:::python
        TLpips["lpipsPyTorch"]:::python
        TS --- T1
        TArg --- T1
        TUtil --- T1
        TLpips --- T1
    end

    %% Differentiable Rasterization
    subgraph "Differentiable Rasterization"
        Raster["Diff Rasterizer"]:::cpp
    end

    %% Loss & Metrics
    subgraph "Loss & Metrics"
        Metrics["metrics.py"]:::python
    end

    %% Rendering and Visualization
    subgraph "Rendering and Visualization"
        GR["gaussian_renderer"]:::ui
        WV["web_viewer"]:::ui
        Render["render.py"]:::python
        FullEval["full_eval.py"]:::python
        GR --- WV
        Render --- FullEval
    end

    %% Main Data Flow
    Scene -->|"SceneData"| T1
    T1 -->|"GaussianParams"| Raster
    Raster -->|"RenderedImage"| T1
    T1 -->|"LossCalc"| Metrics
    Metrics -->|"Feedback"| T1
    T1 -->|"GUIOutput"| GR
    T1 -->|"Websockets"| WV
    T1 -->|"FinalRender"| Render
    T1 -->|"Evaluation"| FullEval

    %% Click Events
    click Scene "https://github.com/humansensinglab/taming-3dgs/tree/main/scene/"
    click T1 "https://github.com/humansensinglab/taming-3dgs/blob/main/train.py"
    click TS "https://github.com/humansensinglab/taming-3dgs/blob/main/train.sh"
    click TArg "https://github.com/humansensinglab/taming-3dgs/tree/main/arguments/"
    click TUtil "https://github.com/humansensinglab/taming-3dgs/tree/main/utils/"
    click TLpips "https://github.com/humansensinglab/taming-3dgs/tree/main/lpipsPyTorch/"
    click Raster "https://github.com/humansensinglab/taming-3dgs/tree/main/submodules/diff-gaussian-rasterization/"
    click Metrics "https://github.com/humansensinglab/taming-3dgs/blob/main/metrics.py"
    click GR "https://github.com/humansensinglab/taming-3dgs/tree/main/gaussian_renderer/"
    click WV "https://github.com/humansensinglab/taming-3dgs/tree/main/web_viewer/"
    click Render "https://github.com/humansensinglab/taming-3dgs/blob/main/render.py"
    click FullEval "https://github.com/humansensinglab/taming-3dgs/blob/main/full_eval.py"

    %% Styles
    classDef python fill:#D1E8FF,stroke:#333,stroke-width:2px
    classDef bash fill:#FDEBD0,stroke:#333,stroke-width:2px
    classDef cpp fill:#FAD7A0,stroke:#333,stroke-width:2px
    classDef ui fill:#F9E79F,stroke:#333,stroke-width:2px
```
