# pierotofy/opensplat architecture

> Architecture diagram of the GitHub repository pierotofy/opensplat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pierotofy/opensplat
- Repository: https://github.com/pierotofy/opensplat

## Overview

1. **What this repository is**
   OpenSplat is a single C++ application/library for turning SfM-style reconstructions into trained 3D Gaussian splat scenes. The end product is a scene file, typically `splat.ply` or compressed `.splat`, plus `cameras.json` for viewers.

2. **Top-level architecture**
   The repo is organized as a pipeline: dataset ingestion → scene/model initialization → iterative optimization → rasterization/rendering → export. Most of the logic lives in C++ source files at the root, with a small number of backend-specific rasterizer implementations under `rasterizer/`.

3. **Input adapters / dataset boundaries**
   The files `colmap.*`, `openmvg.*`, `opensfm.*`, `nerfstudio.*`, and `opensplat.*` indicate separate importers for different project formats. These adapters normalize external reconstruction formats into a shared internal representation, so the rest of the system does not depend on any one photogrammetry tool.

4. **Core scene representation**
   `input_data.*`, `model.*`, `point_io.*`, and `constants.hpp` are the main data-model layer. This is where cameras, sparse points, gaussian parameters, and scene metadata are represented and serialized. `point_io` is the likely bridge for reading/writing point clouds and final splat outputs.

5. **Training / optimization pipeline**
   `simple_trainer.cpp`, `optim_scheduler.*`, `ssim.*`, and `visualizer.*` form the training/control layer. The trainer orchestrates repeated forward passes, loss computation, scheduling, densification/pruning-style updates, and progress reporting. The presence of `ssim` and `spherical_harmonics` suggests image-space reconstruction loss and view-dependent color modeling are central to optimization.

6. **Geometry and math utilities**
   `tensor_math.*`, `kdtree_tensor.*`, `tile_bounds.hpp`, `project_gaussians.*`, and `utils.*` are the computational support layer. They handle projection math, spatial indexing, tile/bin bounds, and tensor operations used by both the trainer and rasterizer. This layer is the main boundary between high-level training code and low-level acceleration code.

7. **Rasterization subsystem**
   `rasterize_gaussians.*` is the abstraction for drawing gaussians into images. The actual implementations live in `rasterizer/gsplat`, `rasterizer/gsplat-cpu`, and `rasterizer/gsplat-metal`, which means rendering is split behind a backend interface. This is the performance-critical part of the system and the main place where GPU/CPU/Metal differences matter.

8. **Backend-specific compute**
   The `rasterizer/gsplat` directory contains CUDA kernels (`forward.cu`, `backward.cu`, `bindings.cu`) and shared headers, showing that NVIDIA GPU support is a first-class path. `rasterizer/gsplat-cpu` provides a CPU fallback, and `rasterizer/gsplat-metal` provides Apple Metal support. The build selects one of these runtime backends, so the application architecture is portable but not runtime-pluggable in the dynamic sense.

9. **Framework and library dependencies**
   The project is built with CMake and links against OpenCV and libtorch. LibTorch supplies tensor storage and autograd-style operations, while OpenCV appears to support image loading and computer vision helpers. That means the training loop is C++/Torch-centric rather than Python-based.

10. **Command-line product boundary**
   The main executable behavior described in the README is: point the program at an existing COLMAP/OpenSfM/OpenMVG/nerfstudio/ODM reconstruction, optionally choose output/compression/resume flags, and emit trained splat assets. There is no requirement for a server, database, or web frontend; this is an offline compute tool with file-based inputs and outputs.

11. **Cross-platform and GPU build matrix**
   The `.github/workflows/*`, multiple `Dockerfile*` variants, and README build sections show the repo is explicitly engineered for Linux, Windows, macOS, CUDA, HIP/ROCm, and CPU-only builds. Architecture-wise, this means platform differences are mostly isolated to build configuration and backend kernels rather than the high-level training pipeline.

12. **Diagram-worthy data flow**
   A useful architecture diagram should show external reconstruction data entering through format-specific importers, being normalized into internal scene/model state, then flowing through training and projection/rasterization kernels, with losses/schedulers updating the gaussian model iteratively. The final outputs are serialized splat assets and camera metadata for downstream viewers/editors.

Main entry points: [`simple_trainer.cpp`](https://github.com/pierotofy/opensplat/blob/main/simple_trainer.cpp) (Trainer), [`README.md`](https://github.com/pierotofy/opensplat/blob/main/README.md) (Pipeline), [`input_data.cpp`](https://github.com/pierotofy/opensplat/blob/main/input_data.cpp) (Input data), [`project_gaussians.cpp`](https://github.com/pierotofy/opensplat/blob/main/project_gaussians.cpp) (Projection)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Input adapters"]
  node_colmap["COLMAP<br/>importer<br/>[colmap.cpp]"]
  node_openmvg["OpenMVG<br/>importer<br/>[openmvg.cpp]"]
  node_opensfm["OpenSfM<br/>importer<br/>[opensfm.cpp]"]
  node_nerfstudio["Nerfstudio<br/>importer<br/>[nerfstudio.cpp]"]
  node_opensplat["OpenSplat IO<br/>importer/exporter<br/>[opensplat.cpp]"]
end

subgraph group_core["Core model"]
  node_input_data["Input data<br/>scene schema<br/>[input_data.cpp]"]
  node_model["Gaussian model<br/>scene state<br/>[model.cpp]"]
  node_point_io["Point I/O<br/>pointcloud<br/>[point_io.cpp]"]
  node_constants["Constants<br/>params<br/>[constants.hpp]"]
end

subgraph group_train["Training loop"]
  node_trainer["Trainer<br/>optimizer<br/>[simple_trainer.cpp]"]
  node_scheduler["Scheduler<br/>control"]
  node_ssim["SSIM<br/>loss<br/>[ssim.cpp]"]
  node_visualizer["Visualizer<br/>reporting<br/>[visualizer.cpp]"]
end

subgraph group_math["Geometry/math"]
  node_tensor_math["Tensor math<br/>compute<br/>[tensor_math.cpp]"]
  node_kdtree["KD-tree<br/>spatial index<br/>[kdtree_tensor.cpp]"]
  node_project["Projection<br/>geometry"]
  node_utils["Utilities<br/>support<br/>[utils.cpp]"]
  node_sph["Spherical harmonics<br/>color basis"]
end

subgraph group_render["Rasterization"]
  node_rasterize["Rasterizer API<br/>render facade"]
  node_gsplat_cuda["CUDA backend<br/>gpu rasterizer<br/>[forward.cu]"]
  node_gsplat_cpu["CPU backend<br/>cpu rasterizer<br/>[gsplat_cpu.cpp]"]
  node_gsplat_metal["Metal backend<br/>metal rasterizer<br/>[gsplat_metal.mm]"]
end

subgraph group_deps["Build targets"]
  node_readme["Pipeline<br/>docs<br/>[README.md]"]
  node_build["CMake build<br/>build system<br/>[CMakeLists.txt]"]
end

node_readme -->|"inputs"| node_colmap
node_readme -->|"inputs"| node_openmvg
node_readme -->|"inputs"| node_opensfm
node_readme -->|"inputs"| node_nerfstudio
node_readme -->|"outputs"| node_opensplat
node_colmap -->|"normalize"| node_input_data
node_openmvg -->|"normalize"| node_input_data
node_opensfm -->|"normalize"| node_input_data
node_nerfstudio -->|"normalize"| node_input_data
node_opensplat -->|"serialize"| node_point_io
node_input_data -->|"init"| node_model
node_point_io -->|"load/save"| node_model
node_model -->|"optimize"| node_trainer
node_scheduler -->|"control"| node_trainer
node_ssim -->|"loss"| node_trainer
node_visualizer -->|"observe"| node_trainer
node_trainer -->|"forward pass"| node_project
node_trainer -->|"render"| node_rasterize
node_project -->|"math"| node_tensor_math
node_project -->|"spatial queries"| node_kdtree
node_project -->|"params"| node_constants
node_trainer -->|"appearance"| node_sph
node_rasterize -->|"backend"| node_gsplat_cuda
node_rasterize -->|"backend"| node_gsplat_cpu
node_rasterize -->|"backend"| node_gsplat_metal
node_gsplat_cuda -->|"compute"| node_tensor_math
node_gsplat_cpu -->|"compute"| node_tensor_math
node_gsplat_metal -->|"compute"| node_tensor_math
node_build -->|"selects"| node_gsplat_cuda
node_build -->|"selects"| node_gsplat_cpu
node_build -->|"selects"| node_gsplat_metal

click node_readme "https://github.com/pierotofy/opensplat/blob/main/README.md"
click node_colmap "https://github.com/pierotofy/opensplat/blob/main/colmap.cpp"
click node_openmvg "https://github.com/pierotofy/opensplat/blob/main/openmvg.cpp"
click node_opensfm "https://github.com/pierotofy/opensplat/blob/main/opensfm.cpp"
click node_nerfstudio "https://github.com/pierotofy/opensplat/blob/main/nerfstudio.cpp"
click node_opensplat "https://github.com/pierotofy/opensplat/blob/main/opensplat.cpp"
click node_input_data "https://github.com/pierotofy/opensplat/blob/main/input_data.cpp"
click node_model "https://github.com/pierotofy/opensplat/blob/main/model.cpp"
click node_point_io "https://github.com/pierotofy/opensplat/blob/main/point_io.cpp"
click node_constants "https://github.com/pierotofy/opensplat/blob/main/constants.hpp"
click node_trainer "https://github.com/pierotofy/opensplat/blob/main/simple_trainer.cpp"
click node_scheduler "https://github.com/pierotofy/opensplat/blob/main/optim_scheduler.cpp"
click node_ssim "https://github.com/pierotofy/opensplat/blob/main/ssim.cpp"
click node_visualizer "https://github.com/pierotofy/opensplat/blob/main/visualizer.cpp"
click node_tensor_math "https://github.com/pierotofy/opensplat/blob/main/tensor_math.cpp"
click node_kdtree "https://github.com/pierotofy/opensplat/blob/main/kdtree_tensor.cpp"
click node_project "https://github.com/pierotofy/opensplat/blob/main/project_gaussians.cpp"
click node_utils "https://github.com/pierotofy/opensplat/blob/main/utils.cpp"
click node_sph "https://github.com/pierotofy/opensplat/blob/main/spherical_harmonics.cpp"
click node_rasterize "https://github.com/pierotofy/opensplat/blob/main/rasterize_gaussians.cpp"
click node_gsplat_cuda "https://github.com/pierotofy/opensplat/blob/main/rasterizer/gsplat/forward.cu"
click node_gsplat_cpu "https://github.com/pierotofy/opensplat/blob/main/rasterizer/gsplat-cpu/gsplat_cpu.cpp"
click node_gsplat_metal "https://github.com/pierotofy/opensplat/blob/main/rasterizer/gsplat-metal/gsplat_metal.mm"
click node_build "https://github.com/pierotofy/opensplat/blob/main/CMakeLists.txt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_colmap,node_openmvg,node_opensfm,node_nerfstudio,node_opensplat toneBlue
class node_input_data,node_model,node_point_io,node_constants toneAmber
class node_trainer,node_scheduler,node_ssim,node_visualizer toneMint
class node_tensor_math,node_kdtree,node_project,node_utils,node_sph toneRose
class node_rasterize,node_gsplat_cuda,node_gsplat_cpu,node_gsplat_metal toneIndigo
class node_readme,node_build toneTeal
```

## Components

### Input adapters

Normalize external SfM/reconstruction formats into the internal scene representation.

- [COLMAP](https://github.com/pierotofy/opensplat/blob/main/colmap.cpp): importer · Importer for COLMAP-style reconstructions. · `colmap.cpp`
- [OpenMVG](https://github.com/pierotofy/opensplat/blob/main/openmvg.cpp): importer · Importer for OpenMVG reconstructions. · `openmvg.cpp`
- [OpenSfM](https://github.com/pierotofy/opensplat/blob/main/opensfm.cpp): importer · Importer for OpenSfM reconstructions. · `opensfm.cpp`
- [Nerfstudio](https://github.com/pierotofy/opensplat/blob/main/nerfstudio.cpp): importer · Importer for Nerfstudio datasets/reconstructions. · `nerfstudio.cpp`
- [OpenSplat IO](https://github.com/pierotofy/opensplat/blob/main/opensplat.cpp): importer/exporter · Native scene format adapter and serialization boundary. · `opensplat.cpp`

### Core model

Scene, camera, point, and export data structures used across the pipeline.

- [Input data](https://github.com/pierotofy/opensplat/blob/main/input_data.cpp): scene schema · Normalized cameras, images, and sparse reconstruction data. · `input_data.cpp`
- [Gaussian model](https://github.com/pierotofy/opensplat/blob/main/model.cpp): scene state · Trainable gaussian scene representation and parameters. · `model.cpp`
- [Point I/O](https://github.com/pierotofy/opensplat/blob/main/point_io.cpp): pointcloud · Point-cloud and splat scene read/write bridge. · `point_io.cpp`
- [Constants](https://github.com/pierotofy/opensplat/blob/main/constants.hpp): params · Shared numeric and structural constants. · `constants.hpp`

### Training loop

Optimization, losses, and progress control for fitting gaussians to images.

- [Trainer](https://github.com/pierotofy/opensplat/blob/main/simple_trainer.cpp): optimizer · Main iterative optimization loop and orchestration. · `simple_trainer.cpp`
- [Scheduler](https://github.com/pierotofy/opensplat/blob/main/optim_scheduler.cpp): control · Learning-rate and training schedule control. · `optim_scheduler.cpp`
- [SSIM](https://github.com/pierotofy/opensplat/blob/main/ssim.cpp): loss · Image-space structural similarity loss component. · `ssim.cpp`
- [Visualizer](https://github.com/pierotofy/opensplat/blob/main/visualizer.cpp): reporting · Progress display and training-time inspection. · `visualizer.cpp`

### Geometry/math

Projection, tensor, spatial indexing, and shared computation helpers.

- [Tensor math](https://github.com/pierotofy/opensplat/blob/main/tensor_math.cpp): compute · Shared tensor operations and numeric helpers. · `tensor_math.cpp`
- [KD-tree](https://github.com/pierotofy/opensplat/blob/main/kdtree_tensor.cpp): spatial index · Spatial acceleration structure for neighborhoods and selection. · `kdtree_tensor.cpp`
- [Projection](https://github.com/pierotofy/opensplat/blob/main/project_gaussians.cpp): geometry · Projects gaussians into image space and tile bounds. · `project_gaussians.cpp`
- [Utilities](https://github.com/pierotofy/opensplat/blob/main/utils.cpp): support · Common helpers used across training and rendering. · `utils.cpp`
- [Spherical harmonics](https://github.com/pierotofy/opensplat/blob/main/spherical_harmonics.cpp): color basis · View-dependent appearance representation. · `spherical_harmonics.cpp`

### Rasterization

Backend-agnostic rasterization entrypoints and backend-specific implementations.

- [Rasterizer API](https://github.com/pierotofy/opensplat/blob/main/rasterize_gaussians.cpp): render facade · Backend-agnostic entrypoint for gaussian rasterization. · `rasterize_gaussians.cpp`
- [CUDA backend](https://github.com/pierotofy/opensplat/blob/main/rasterizer/gsplat/forward.cu): gpu rasterizer · CUDA forward/backward kernels and bindings. · `rasterizer/gsplat/forward.cu`
- [CPU backend](https://github.com/pierotofy/opensplat/blob/main/rasterizer/gsplat-cpu/gsplat_cpu.cpp): cpu rasterizer · CPU rasterization implementation. · `rasterizer/gsplat-cpu/gsplat_cpu.cpp`
- [Metal backend](https://github.com/pierotofy/opensplat/blob/main/rasterizer/gsplat-metal/gsplat_metal.mm): metal rasterizer · Apple Metal rasterization implementation. · `rasterizer/gsplat-metal/gsplat_metal.mm`

### Build targets

Build system and platform-specific automation for supported runtimes.

- [Pipeline](https://github.com/pierotofy/opensplat/blob/main/README.md): docs · Offline workflow from reconstruction input to trained splat outputs and camera metadata. · `README.md`
- [CMake build](https://github.com/pierotofy/opensplat/blob/main/CMakeLists.txt): build system · Top-level build configuration and dependency wiring. · `CMakeLists.txt`

## Connections

- Pipeline → COLMAP: inputs · External reconstructions enter through format-specific importers.
- Pipeline → OpenMVG: inputs · External reconstructions enter through format-specific importers.
- Pipeline → OpenSfM: inputs · External reconstructions enter through format-specific importers.
- Pipeline → Nerfstudio: inputs · External reconstructions enter through format-specific importers.
- Pipeline → OpenSplat IO: outputs · Trained splat assets and camera metadata are emitted for viewers/editors.
- COLMAP → Input data: normalize · Importer converts source format into shared scene data.
- OpenMVG → Input data: normalize · Importer converts source format into shared scene data.
- OpenSfM → Input data: normalize · Importer converts source format into shared scene data.
- Nerfstudio → Input data: normalize · Importer converts source format into shared scene data.
- OpenSplat IO → Point I/O: serialize · Native format is written and read through point/splat I/O.
- Input data → Gaussian model: init · Scene state seeds gaussian model initialization.
- Point I/O → Gaussian model: load/save · Model parameters and splat data pass through point I/O.
- Gaussian model → Trainer: optimize · Trainer updates the gaussian scene model iteratively.
- Scheduler → Trainer: control · Scheduling steers optimization dynamics.
- SSIM → Trainer: loss · Image-space objective contributes to training feedback.
- Visualizer → Trainer: observe · Training progress is reported and inspected.
- Trainer → Projection: forward pass · Training projects gaussians into camera views.
- Trainer → Rasterizer API: render · Forward rendering feeds reconstruction loss computation.
- Projection → Tensor math: math · Projection uses shared numeric and tensor operations.
- Projection → KD-tree: spatial queries · Spatial indexing supports neighborhood and visibility computations.
- Projection → Constants: params · Projection math relies on shared constants.
- Trainer → Spherical harmonics: appearance · View-dependent color modeling uses spherical harmonics.
- Rasterizer API → CUDA backend: backend · CUDA implementation provides the fast path.
- Rasterizer API → CPU backend: backend · CPU implementation provides a fallback path.
- Rasterizer API → Metal backend: backend · Metal implementation provides Apple platform support.
- CUDA backend → Tensor math: compute · Backend kernels depend on shared tensor/math helpers.
- CPU backend → Tensor math: compute · Backend code reuses shared tensor/math helpers.
- Metal backend → Tensor math: compute · Backend code reuses shared tensor/math helpers.
- CMake build → CUDA backend: selects · Build configuration chooses the active backend.
- CMake build → CPU backend: selects · Build configuration chooses the active backend.
- CMake build → Metal backend: selects · Build configuration chooses the active backend.
