# yzslab/gaussian-splatting-lightning architecture

> Architecture diagram of the GitHub repository yzslab/gaussian-splatting-lightning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yzslab/gaussian-splatting-lightning
- Repository: https://github.com/yzslab/gaussian-splatting-lightning

## Overview

##### Project Type and Purpose
- It is not a typical full-stack web application but rather a research/development system focused on 3D scene reconstruction with advanced rendering techniques (e.g., Gaussian splatting, deformable models, mip-splatting, appearance embedding, etc.).
- Its purpose is to provide multi-GPU training, support a variety of datasets (Blender, Colmap, PolyCam, etc.), and to deliver diverse rendering pipelines (including a web-based viewer). The README makes it clear that the system supports training routines, evaluation, and interactive viewing.

##### Analyzing the File Structure
- Key top-level directories like “configs”, “internal”, “utils”, “notebooks” and “tests” reveal a modular, layered design.
- Under “internal”, you have subdirectories for critical components:
- “dataparsers”: Handles various dataset inputs (e.g., Colmap, Blender, NERVF datasets).
- “models” and “model_components”: Contain the core Gaussian models and their variants (e.g., vanilla, deformable, appearance models).
- “renderers”: Implements multiple rendering strategies (e.g., gsplat, vanilla, deformable, distributed renderers, etc.).
- “density_controllers”: Contains modules to manage density, including specialized controllers like MCMC, static, or taming-based ones.
- “configs”: Contains Python scripts to interpret and manage training configuration.
- “utils”: Houses scripts for dataset preprocessing, conversion (e.g., ckpt2ply, image downsampling), merging model partitions, etc.
- “viewer”: Provides an interactive web view layer powered by a web viewer library “viser” for model visualization.
- “main.py” and “render.py” are the entry points for training and rendering flows.
- The “notebooks” folder shows several exploratory and preprocessing pipelines, while “tests” provide unit tests for core functionality.

##### Key Architectural Components
a. Data Handling & Preprocessing
- Data input is managed by multiple dataparsers (located in internal/dataparsers) that support different dataset types (Colmap, Blender, PolyCam, etc.).
- Preprocessing scripts (in utils) handle operations like downsampling images, generating appearance groups, estimating depths, and converting models (e.g., ckpt2ply).

b. Configuration / Hyperparameter Management
- A large collection of YAML config files in the “configs” folder (with various sub-categories) that dictate training parameters, model architecture, rendering options, and multi-GPU settings.
- Internal configuration modules (found in internal/configs and utils/scalable_param_configs) allow dynamic adjustment of hyperparameters.

c. Model Components & Rendering Pipelines
- “internal/models” and “internal/model_components” incorporate various Gaussian splatting methods (basic Gaussian, deformable, periodic vibration, etc.).
- “internal/renderers” contains distinct renderers that implement different strategies:
- Standard GSplat renderers
- Appearance embedding renderers (both view-dependent and independent versions)
- Distributed renderers for multi-GPU training
- Variants for specific derived algorithms (Mip-Splatting, Feature Distillation, etc.)
- Density controllers (internal/density_controllers) modulate the evolution and densification of Gaussians during training.

d. Training & Optimization
- The training pipeline is orchestrated from main.py, which integrates:
- PyTorch Lightning for scheduled training loops.
- Multi-GPU strategies (both standard DDP and a newer distributed approach for densification).
- Logging, callbacks, and metric computations (as seen in internal/metrics).

e. Post-Processing & Evaluation
- Tools for converting checkpoints to ply, merging distributed checkpoints (utils/merge_distributed_ckpts.py), and for rendering videos.
- Evaluation modules in main.py (and separate commands) to compute image-based metrics (PSNR, SSIM, LPIPS).

f. Interactive Visualization
- The “viewer” module (internal/viewer and viewer.py) provides a web-based interactive UI allowing users to load one or more models, adjust camera transforms, and view results in real-time.

##### Relationships and Data Flow
- Start at the Data Layer: The dataset is preprocessed (via utils or notebooks) and loaded using a specific dataparser.
- The parsed data is then fed into the Training Pipeline (main.py) where configurations from YAML files determine which model components and density controllers to instantiate.
- The results (trained checkpoints) may be post-processed (eg, converted to ply) and optionally loaded in the web viewer.
- Multi-GPU modules alter data loading (distributing subsets to each process) and coordinate synchronized updates via Lightning’s DDP or custom strategies.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input Layer
    subgraph "Input Layer"
        A1["Dataparsers"]:::data
        A2["Preprocessing Scripts"]:::data
        A3["Exploratory Notebooks"]:::data
    end

    %% Core Engine
    subgraph "Core Engine"
        %% Configuration Manager split into three parts
        B0A["YAML Config Files"]:::core
        B0B["Internal Config Modules"]:::core
        B0C["Scalable Parameter Configs"]:::core

        %% Model Components split into Models and Model Components
        B3a["Models"]:::core
        B3b["Model Components"]:::core

        B4["Renderers"]:::core
        B5["Density Controllers"]:::core
        B6["Multi-GPU & Optimization"]:::core
        B1["main.py (Training Engine)"]:::core
    end

    %% Output Layer
    subgraph "Output Layer"
        C1["Post-Processing & Evaluation"]:::output
        C2["Interactive Visualization"]:::output
    end

    %% Utility Modules
    subgraph "Utility Modules"
        D1["General Utilities"]:::util
        D2["Additional Scripts"]:::util
    end

    %% Data Flow from Input to Core Engine
    A1 -->|"InputData"| B1
    A2 -->|"InputData"| B1
    A3 -->|"InputData"| B1

    %% Configuration influences Model, Renderer, and Density setups
    B0A -->|"Configures"| B3a
    B0A -->|"Configures"| B4
    B0A -->|"Configures"| B5

    B0B -->|"Configures"| B3a
    B0B -->|"Configures"| B4
    B0B -->|"Configures"| B5

    B0C -->|"Configures"| B3a
    B0C -->|"Configures"| B4
    B0C -->|"Configures"| B5

    %% Model Components contribution to Training Engine
    B3a -->|"ModelFeed"| B1
    B3b -->|"ComponentFeed"| B1
    B4 -->|"Rendering"| B1
    B5 -->|"DensityControl"| B1
    B6 -->|"MultiGPU"| B1

    %% Training Engine flows to Output Layer
    B1 -->|"TrainedModels"| C1
    B1 -->|"LiveOutput"| C2

    %% Utility support arrows
    D1 -->|"PreprocessingSupport"| A2
    D1 -->|"CoreUtility"| B1
    D2 -->|"ScriptSupport"| B1

    %% Click Events
    click A1 "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/internal/dataparsers"
    click A2 "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/utils/"
    click A3 "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/notebooks/"

    click B0A "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/configs/"
    click B0B "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/internal/configs/"
    click B0C "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/utils/scalable_param_configs/"

    click B3a "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/internal/models/"
    click B3b "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/internal/model_components/"
    click B4 "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/internal/renderers/"
    click B5 "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/internal/density_controllers/"
    click B6 "https://github.com/yzslab/gaussian-splatting-lightning/blob/main/internal/mp_strategy.py"
    click B1 "https://github.com/yzslab/gaussian-splatting-lightning/blob/main/main.py"

    click C1 "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/utils/"
    click C2 "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/internal/viewer/"

    click D1 "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/internal/utils/"
    click D2 "https://github.com/yzslab/gaussian-splatting-lightning/tree/main/scripts/"

    %% Styling
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef core fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef output fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef util fill:#FFFFE0,stroke:#000,stroke-width:2px;
```
