# szymanowiczs/splatter-image architecture

> Architecture diagram of the GitHub repository szymanowiczs/splatter-image, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/szymanowiczs/splatter-image
- Repository: https://github.com/szymanowiczs/splatter-image

## Overview

##### Main Components
- Configuration & Experiment Management:
- Folder: configs (with subdirectories for camera embeddings, datasets, experiments, Hydra, WandB)
- Purpose: Holds all hyper-parameter configurations, experiment-specific settings, and defaults.
- Data Preprocessing & Dataset Management:
- Folder: data_preprocessing (e.g. preprocess_co3d.py)
- Folder: datasets (multiple dataset reader modules for CO3D, SRN, Objaverse, etc.)
- Purpose: Downloads, preprocesses, and provides input data in various formats for different datasets.
- Model & Training:
- Files: train_network.py, scene/gaussian_predictor.py
- Purpose: The training loop (two-stage training with and without LPIPS fine-tuning) and network definition.
- Rendering:
- Folder: gaussian_renderer
- Purpose: Implements the Gaussian Splatting renderer to produce images from the 3D representation.
- Evaluation & Demo:
- Files: eval.py, gradio_app.py, demo_examples (holds demo videos)
- Purpose: Evaluation routines for multiple datasets, and an interactive demo (using Gradio) for user-upload of images and showcasing the results.
- Utilities & Testing:
- Folder: utils (general helper functions: camera, graphics, loss, etc.)
- Folder: tests (unit tests for key parts)
- Purpose: Provide supporting functionality and testing frameworks.

##### Relationships and Interactions
- The Configurations (configs) are used by almost every component (training, evaluation, preprocessing) to set experiment parameters and external service options (like WandB logging).
- During Data Preprocessing, raw datasets (CO3D, ShapeNet, Objaverse, GSO) are processed by scripts under data_preprocessing, and dataset modules (in datasets/) load and prepare data for training/evaluation.
- The Training Loop (in train_network.py) uses dataset loaders to feed data into the model defined in scene/gaussian_predictor.py, which in turn makes calls to the Gaussian renderer to convert the implicit 3D reconstruction into rendered images.
- The Evaluation module (eval.py) leverages the same pipeline to load a pretrained model and run validations, optionally also saving rendered visualizations using camera trajectories as implemented in the rendering and utility scripts.
- The Demo Application (gradio_app.py) provides an interactive front-end that wraps the inference/evaluation pipeline to allow users to upload images and view outputs. The demo_examples folder holds example outputs and videos, illustrating results.
- Utility modules, such as those in utils, support all layers (data handling, camera transformations, image processing, etc.), ensuring integration consistency across components.

##### Architectural Patterns & Principles
- Modularity: Clear separation between data handling (preprocessing and dataset loading), model training (training and evaluation), and visualization/rendering.
- Config-driven Execution: Use of YAML files in configs, as well as Hydra for flexible configuration management.
- Two-Stage Training: The training is clearly separated into a primary training stage and a fine-tuning stage using LPIPS, which can be highlighted as distinct modules or phases.
- External Integration: The system integrates with external libraries (PyTorch, PyTorch3D, Gaussian Splatting renderer, Gradio, WandB) – each of which plays a significant role.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Configuration Layer"
        CONFIG["Configurations & Experiment Management"]:::config
    end

    subgraph "Data Ingestion"
        DP["Data Preprocessing (data_preprocessing/)"]:::data
        DS["Datasets (datasets/)"]:::data
    end

    subgraph "Model & Training"
        TN["Training Loop (train_network.py)"]:::training
        GP["Gaussian Predictor (scene/gaussian_predictor.py)"]:::training
    end

    subgraph "Rendering"
        GR["Gaussian Renderer (gaussian_renderer/)"]:::rendering
    end

    subgraph "Evaluation & Demo"
        EV["Evaluation Script (eval.py)"]:::interface
        GD["Gradio Demo (gradio_app.py)"]:::interface
        DE["Demo Examples (demo_examples/)"]:::interface
    end

    subgraph "Utilities & Testing"
        UT["Utilities (utils/)"]:::utility
        TS["Tests (tests/)"]:::utility
    end

    CONFIG -->|"uses"| DP
    CONFIG -->|"configures"| TN
    CONFIG -->|"configures"| EV
    CONFIG -->|"supports"| UT

    DP -->|"prepares"| DS
    DP -->|"feeds"| TN
    DS -->|"loads"| TN

    TN -->|"triggers"| GP
    TN -->|"feeds"| EV
    TN -->|"enables"| GD

    TN <-->|"renders"| GR
    GP <-->|"renders"| GR

    EV -->|"informs"| GD

    UT -->|"supports"| DP
    UT -->|"supports"| TN
    UT -->|"supports"| EV

    classDef config fill:#fceabb,stroke:#e8b23c,stroke-width:2px;
    classDef data fill:#c1f0c1,stroke:#36a162,stroke-width:2px;
    classDef training fill:#cce5ff,stroke:#2869a6,stroke-width:2px;
    classDef rendering fill:#ffd1dc,stroke:#ff69b4,stroke-width:2px;
    classDef interface fill:#ffe6cc,stroke:#e69900,stroke-width:2px;
    classDef utility fill:#e2e2e2,stroke:#888,stroke-width:2px;

    click CONFIG "https://github.com/szymanowiczs/splatter-image/tree/main/configs/"
    click DP "https://github.com/szymanowiczs/splatter-image/tree/main/data_preprocessing/"
    click DS "https://github.com/szymanowiczs/splatter-image/tree/main/datasets/"
    click TN "https://github.com/szymanowiczs/splatter-image/blob/main/train_network.py"
    click GP "https://github.com/szymanowiczs/splatter-image/blob/main/scene/gaussian_predictor.py"
    click GR "https://github.com/szymanowiczs/splatter-image/tree/main/gaussian_renderer/"
    click EV "https://github.com/szymanowiczs/splatter-image/blob/main/eval.py"
    click GD "https://github.com/szymanowiczs/splatter-image/blob/main/gradio_app.py"
    click DE "https://github.com/szymanowiczs/splatter-image/tree/main/demo_examples/"
    click UT "https://github.com/szymanowiczs/splatter-image/tree/main/utils/"
    click TS "https://github.com/szymanowiczs/splatter-image/tree/main/tests/"
```
