# esw0116/odgs architecture

> Architecture diagram of the GitHub repository esw0116/odgs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/esw0116/odgs
- Repository: https://github.com/esw0116/odgs

## Overview

##### Project Type and Purpose
- ODGS is a research‐oriented full-stack application for per‐scene optimization of 3D Gaussian Splatting models from omnidirectional images.
- It ingests a sequence of 360° images plus camera parameters (from SfM), builds a 3D Gaussian representation of the scene, renders it differentiably via a custom CUDA rasterizer, and optimizes the model end‐to‐end with PyTorch.

##### Top‐Level Architecture Components
a. Data Ingestion & Preprocessing
- scene/ and scene_perspective/: Python modules for loading images, poses, intrinsic/extrinsic parameters, Gaussian initialization.
- arguments/: CLI parsing for datasets, hyperparameters.
- submodules/simple-knn: Nearest-neighbor queries used in Gaussian parameter updates.
b. Core 3D Gaussian Engine
- gaussian_renderer/: Python wrapper classes (network_gui.py, pinhole_renderer.py) that assemble and dispatch rendering jobs to CUDA.
- submodules/odgs-gaussian-rasterization/: C++/CUDA rasterizer for forward & backward splatting operations; built via setup.py and CMake.
c. Neural Loss & Optimization
- lpipsPyTorch/: Perceptual loss modules (LPIPS) for photometric quality.
- utils/: Utility functions for camera transforms, graphics math (via glm), loss definitions, system wrappers.
- train.py: Top‐level PyTorch training loop—builds scene model, invokes forward renders, computes losses (MSE, LPIPS), and updates Gaussian parameters.
d. Evaluation & Rendering
- render.py & render_perspective.py: Standalone scripts to produce novel‐view renders after training.
e. Environment & Dependencies
- environment.yaml: Conda specification (Python, PyTorch, CUDA).
- submodules/: External libraries (GLM, STBI image I/O, simple‐knn).

##### Relationships & Data Flow
1. User invokes train.py with dataset path →
2. arguments parses flags → scene loaders read images & poses →
3. GaussianModel initialized (gaussian_model.py) →
4. train loop:
- gaussian_renderer calls CUDA rasterizer forward to produce synthesized images;
- loss_utils computes photometric & perceptual losses (LPIPSPyTorch);
- backward pass flows through CUDA ops → gradients back to Gaussian parameters;
5. Periodic evaluation: render_perspective.py uses trained model to render views.

##### Architectural Patterns and Principles
- Modularity: clear separation between data loaders, model definition, renderer, losses, and training logic.
- Hybrid CPU/GPU pipeline: Python controls high-level flow; heavy lifting (splatting) in C++/CUDA.
- Extensibility: external modules (LPIPS, simple-knn, GLM) plugged in via Python interfaces.
- MVC‐style:
- Model = 3D Gaussian parameters;
- View = Rendering engine;
- Controller = train.py orchestrating optimization.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        args["CLI Arguments"]:::data
        scene["Omnidirectional Loader & Gaussian Init"]:::data
        scene_p["Perspective Loader & Gaussian Init"]:::data
        simpleknn["simple-knn"]:::external
    end

    %% Model Layer
    subgraph "Model Layer"
        gmodel["GaussianModel"]:::model
    end

    %% Rendering Engine
    subgraph "Rendering Engine"
        omniGUI["network_gui.py"]:::render
        pinRenderer["pinhole_renderer.py"]:::render
        subgraph "C++/CUDA"
            rasterizer["CUDA Rasterizer (forward/backward)"]:::render
        end
    end

    %% Loss & Optimization
    subgraph "Loss & Optimization"
        LPIPS["LPIPS Module"]:::loss
        utils["Utils & Loss Utils"]:::loss
        train["train.py"]:::loss
    end

    %% Output
    subgraph "Output"
        render["render.py"]:::output
        renderp["render_perspective.py"]:::output
        checkpoints["Trained Checkpoints"]:::output
    end

    %% External Dependencies
    subgraph "External Dependencies" 
        GLM["GLM, STBI, etc."]:::external
    end

    %% Data Flow
    args -->|"parse flags"| scene
    args -->|"parse flags"| scene_p
    scene -->|"omni images + poses"| gmodel
    scene_p -->|"persp images + poses"| gmodel
    simpleknn -.->|"NN queries"| gmodel

    gmodel -->|"init params"| omniGUI
    gmodel -->|"init params"| pinRenderer
    omniGUI -->|"dispatch render"| rasterizer
    pinRenderer -->|"dispatch render"| rasterizer

    rasterizer -->|"rendered image"| synth["Synthesized Images"]:::data
    scene -->|"ground-truth images"| LPIPS
    synth -->|"synthesized image"| LPIPS

    LPIPS -->|"perceptual loss"| train
    utils -->|"loss functions & transforms"| train
    train -->|"gradients"| gmodel

    train -->|"periodic render"| render
    train -->|"periodic render"| renderp
    gmodel -->|"final model"| checkpoints

    GLM -.-> rasterizer
    GLM -.-> utils

    %% Click Events
    click args "https://github.com/esw0116/odgs/blob/main/arguments/__init__.py"
    click scene "https://github.com/esw0116/odgs/blob/main/scene/__init__.py"
    click scene_p "https://github.com/esw0116/odgs/blob/main/scene_perspective/__init__.py"
    click simpleknn "https://github.com/esw0116/odgs/tree/main/submodules/simple-knn"
    click omniGUI "https://github.com/esw0116/odgs/blob/main/gaussian_renderer/network_gui.py"
    click pinRenderer "https://github.com/esw0116/odgs/blob/main/gaussian_renderer/pinhole_renderer.py"
    click rasterizer "https://github.com/esw0116/odgs/tree/main/submodules/odgs-gaussian-rasterization/"
    click LPIPS "https://github.com/esw0116/odgs/blob/main/lpipsPyTorch/modules/lpips.py"
    click utils "https://github.com/esw0116/odgs/blob/main/utils/camera_utils.py"
    click train "https://github.com/esw0116/odgs/blob/main/train.py"
    click render "https://github.com/esw0116/odgs/blob/main/render.py"
    click renderp "https://github.com/esw0116/odgs/blob/main/render_perspective.py"

    %% Styles
    classDef data fill:#87CEEB,stroke:#000,stroke-width:1px
    classDef model fill:#98FB98,stroke:#000,stroke-width:1px
    classDef render fill:#FFA500,stroke:#000,stroke-width:1px
    classDef loss fill:#FF6347,stroke:#000,stroke-width:1px
    classDef output fill:#9370DB,stroke:#000,stroke-width:1px
    classDef external fill:#A9A9A9,stroke:#000,stroke-width:1px,stroke-dasharray: 5 5
```
