# srameo/le3d architecture

> Architecture diagram of the GitHub repository srameo/le3d, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/srameo/le3d
- Repository: https://github.com/srameo/le3d

## Overview

##### Main Components
- Data & Preprocessing:
– Represent the raw image input and metadata extraction pipelines.
– Highlight the “basicgs/data” directory (which contains utilities such as camera_utils, colmap_dataset, colmap_utils, and prefetch_dataloader) and the “datasets” folder.
– Include scripts such as “scripts/data/extract_exif_as_json.sh” and “scripts/data/local_colmap.sh” for exif extraction and COLMAP-based camera calibration.

- 3D Gaussian Modules:
– Represent the core 3D Gaussian Splatting engine. This lives under “basicgs/gaussians”.
– Show submodules inside “basicgs/gaussians/util” (e.g., graphics, op, ply, render, sh, transform) and highlight that they implement the mathematical and rendering operations needed for 3D splatting.
– Include the different architectural variations such as “color_mlp_arch.py” and “le3d_arch.py.”

- Losses & Metrics:

- Pipelines & Training:
– Show the training flows via “basicgs/pipelines” (which includes pipelines such as “base_pipeline.py”, “gs_base_pipeline.py”, “le3d_pipeline.py” and a learning rate scheduler).
– Highlight the entry points “basicgs/train.py” and “basicgs/test.py” as scripts that start the training and evaluation.

- Utilities & Configuration:
– Note the “options” directory, where YAML configuration files (for different scenes or variants like “le3d” and “vanilla_gs”) are used to customize training/execution.

- Viewing & Editing:
– Represent the interactive components in “basicgs/view” and “basicgs/viewers”.
– Include both the base viewer and specialized editors (e.g., “editor.py”, “le3d_viewer.py”) and their associated utilities (like export_util, frame_util, interpolate_util).
– Connect these components to the post-training export tool “scripts/export_splat.sh” which converts output point cloud (.ply) files to splat files for the web viewer.

- External/3rd-Party Submodules:
– Highlight the “submodules” directory. It includes independent, externally maintained libraries for different aspects of rasterization (like base-rasterization, full-rasterization, gcnt-rasterization, hdr-splat, and hist-rasterization). These are key for achieving GPU-accelerated rendering.
– Also note that this project borrows code from BasicSR and gaussian-splatting projects, so indicate these as external dependencies that integrate with the core system.

##### Relationships and Interaction Flows
- Data Pipeline Flow:
– Raw input images and DNG files are first processed using the data utilities and scripts to extract metadata and calibrate camera positions.
– The processed data flows into the training pipeline.

- Training Flow:
– The training pipeline (configured via YAML files in “options”) loads data, builds models using modules defined in “basicgs/gaussians”, applies loss functions (from “basicgs/losses”), and computes metrics (from “basicgs/metrics”).
– Utilities help manage options and logging.

- Rendering & Viewing Flow:
– Once training is complete, a point cloud (PLY file) is generated.
– This is then converted into a rendered output (splat file) using provided export scripts.

- External Rendering Submodules:
– The GPU-accelerated rasterization submodules integrate with the core code to perform real-time display and render computations.

##### Architectural Patterns and Design Principles
- Modularity: The code is organized into discrete modules (data, gaussians, losses, pipelines, viewers, utils) to separate concerns.
- Configuration-driven: The “options” directory and YAML files drive many aspects of training and inference.
- Extensibility: The use of registries (such as in “basicgs/utils/registry.py”) suggests a plugin-like architecture that enables easy extension.
- Real-Time Rendering Considerations: Use of GPU submodules for rasterization and CUDA kernels in submodules indicates optimized real-time rendering.

##### Relevant Technologies and Frameworks
- Python as the main programming language.
- Pytorch for deep learning and training the 3D Gaussian models.
- CUDA for GPU acceleration (with scripts for installing CUDA support and inferred use of custom CUDA kernels in submodules).
- COLMAP for camera calibration and possibly for sparse reconstruction.
- Integration with BasicSR style framework and reliance on external libraries from “submodules” for advanced rasterization.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data & Preprocessing"
        DATA1["basicgs/data"]:::data
        DS["datasets"]:::data
        SCRIPTS_DATA["scripts/data"]:::scripts
    end

    subgraph "3D Gaussian Modules"
        GAUSS["basicgs/gaussians"]:::core
        GAUSS_UTIL["basicgs/gaussians/util"]:::core
        ARCH1["color_mlp_arch.py"]:::submodule
        ARCH2["le3d_arch.py"]:::submodule
    end

    subgraph "Losses & Metrics"
        LOSSES["basicgs/losses"]:::loss
        METRICS["basicgs/metrics"]:::metrics
    end

    subgraph "Pipelines & Training"
        PIPELINES["basicgs/pipelines"]:::pipeline
        TRAIN["basicgs/train.py"]:::entry
        TEST["basicgs/test.py"]:::entry
        OPTIONS1["options"]:::config
    end

    subgraph "Utilities & Configuration"
        UTILS["basicgs/utils"]:::util
        OPTIONS2["options"]:::config
    end

    subgraph "Viewing & Editing"
        VIEWER["basicgs/view.py"]:::viewer
        VIEWERS["basicgs/viewers"]:::viewer
        VIEWERS_UTIL["basicgs/viewers/utils"]:::viewer
        EXPORT["scripts/export_splat.sh"]:::scripts
    end

    subgraph "External Submodules"
        SUBMODULES["submodules"]:::external
    end

    INSTALL["Install & Setup Process"]:::utils

    %% Data Ingestion Flow
    DATA1 -- "preprocess" --> PIPELINES
    DS -- "provideData" --> PIPELINES
    SCRIPTS_DATA -- "extractMetadata" --> PIPELINES

    %% Training Pipeline Flow
    PIPELINES -- "initiateTraining" --> TRAIN
    PIPELINES -- "initiateEval" --> TEST
    OPTIONS1 -- "configures" --> PIPELINES
    OPTIONS2 -- "configures" --> UTILS

    %% 3D Gaussian Model Building
    PIPELINES -- "buildModel" --> GAUSS
    GAUSS -- "utilizes" --> GAUSS_UTIL
    GAUSS_UTIL -- "calls" --> ARCH1
    GAUSS_UTIL -- "calls" --> ARCH2

    %% Loss and Metrics Integration
    PIPELINES -- "computeLoss" --> LOSSES
    PIPELINES -- "computeMetrics" --> METRICS

    %% Utilities Support
    TRAIN -- "logAndDebug" --> UTILS
    TEST -- "logAndDebug" --> UTILS

    %% Rendering and Viewing Flow
    PIPELINES -- "postProcess" --> EXPORT
    EXPORT -- "renderOutput" --> VIEWER
    EXPORT -- "renderOutput" --> VIEWERS

    %% External GPU-Accelerated Rendering
    GAUSS -- "acceleratedRender" --> SUBMODULES

    %% Installation Process Connections
    INSTALL --- DATA1
    INSTALL --- GAUSS
    INSTALL --- UTILS
    INSTALL --- SUBMODULES

    %% Click Events
    click DATA1 "https://github.com/srameo/le3d/tree/main/basicgs/data"
    click DS "https://github.com/srameo/le3d/tree/main/datasets"
    click SCRIPTS_DATA "https://github.com/srameo/le3d/tree/main/scripts/data"
    click GAUSS "https://github.com/srameo/le3d/tree/main/basicgs/gaussians"
    click GAUSS_UTIL "https://github.com/srameo/le3d/tree/main/basicgs/gaussians/util"
    click ARCH1 "https://github.com/srameo/le3d/blob/main/basicgs/gaussians/color_mlp_arch.py"
    click ARCH2 "https://github.com/srameo/le3d/blob/main/basicgs/gaussians/le3d_arch.py"
    click LOSSES "https://github.com/srameo/le3d/tree/main/basicgs/losses"
    click METRICS "https://github.com/srameo/le3d/tree/main/basicgs/metrics"
    click PIPELINES "https://github.com/srameo/le3d/tree/main/basicgs/pipelines"
    click TRAIN "https://github.com/srameo/le3d/blob/main/basicgs/train.py"
    click TEST "https://github.com/srameo/le3d/blob/main/basicgs/test.py"
    click OPTIONS1 "https://github.com/srameo/le3d/tree/main/options"
    click OPTIONS2 "https://github.com/srameo/le3d/tree/main/options"
    click UTILS "https://github.com/srameo/le3d/tree/main/basicgs/utils"
    click VIEWER "https://github.com/srameo/le3d/blob/main/basicgs/view.py"
    click VIEWERS "https://github.com/srameo/le3d/tree/main/basicgs/viewers"
    click VIEWERS_UTIL "https://github.com/srameo/le3d/tree/main/basicgs/viewers/utils"
    click EXPORT "https://github.com/srameo/le3d/blob/main/scripts/export_splat.sh"
    click SUBMODULES "https://github.com/srameo/le3d/tree/main/submodules"

    %% Styles
    classDef data fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef scripts fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef core fill:#A3E4D7,stroke:#1ABC9C,stroke-width:2px;
    classDef submodule fill:#F5CBA7,stroke:#D35400,stroke-width:2px;
    classDef loss fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
    classDef metrics fill:#FAD7A0,stroke:#E67E22,stroke-width:2px;
    classDef pipeline fill:#D2B4DE,stroke:#8E44AD,stroke-width:2px;
    classDef entry fill:#F7DC6F,stroke:#B7950B,stroke-width:2px;
    classDef config fill:#D0ECE7,stroke:#16A085,stroke-width:2px;
    classDef util fill:#FADBD8,stroke:#E74C3C,stroke-width:2px;
    classDef viewer fill:#D6EAF8,stroke:#3498DB,stroke-width:2px;
    classDef external fill:#FDEDEC,stroke:#C0392B,stroke-width:2px;
    classDef utils fill:#EDBB99,stroke:#A04000,stroke-width:2px;
```
