How pierotofy/opensplat works
1. What this repository is
OpenSplat is a single C++ application/library for turning SfM-style reconstructions into trained 3D Gaussian splat scenes. The end product is a scene file, typically splat.ply or compressed .splat, plus cameras.json for viewers.
2. Top-level architecture
The repo is organized as a pipeline: dataset ingestion → scene/model initialization → iterative optimization → rasterization/rendering → export. Most of the logic lives in C++ source files at the root, with a small number of backend-specific rasterizer implementations under rasterizer/.
3. Input adapters / dataset boundaries
The files colmap.*, openmvg.*, opensfm.*, nerfstudio.*, and opensplat.* indicate separate importers for different project formats. These adapters normalize external reconstruction formats into a shared internal representation, so the rest of the system does not depend on any one photogrammetry tool.
4. Core scene representation
input_data.*, model.*, point_io.*, and constants.hpp are the main data-model layer. This is where cameras, sparse points, gaussian parameters, and scene metadata are represented and serialized. point_io is the likely bridge for reading/writing point clouds and final splat outputs.
5. Training / optimization pipeline
simple_trainer.cpp, optim_scheduler.*, ssim.*, and visualizer.* form the training/control layer. The trainer orchestrates repeated forward passes, loss computation, scheduling, densification/pruning-style updates, and progress reporting. The presence of ssim and spherical_harmonics suggests image-space reconstruction loss and view-dependent color modeling are central to optimization.
6. Geometry and math utilities
tensor_math.*, kdtree_tensor.*, tile_bounds.hpp, project_gaussians.*, and utils.* are the computational support layer. They handle projection math, spatial indexing, tile/bin bounds, and tensor operations used by both the trainer and rasterizer. This layer is the main boundary between high-level training code and low-level acceleration code.
7. Rasterization subsystem
rasterize_gaussians.* is the abstraction for drawing gaussians into images. The actual implementations live in rasterizer/gsplat, rasterizer/gsplat-cpu, and rasterizer/gsplat-metal, which means rendering is split behind a backend interface. This is the performance-critical part of the system and the main place where GPU/CPU/Metal differences matter.
8. Backend-specific compute
The rasterizer/gsplat directory contains CUDA kernels (forward.cu, backward.cu, bindings.cu) and shared headers, showing that NVIDIA GPU support is a first-class path. rasterizer/gsplat-cpu provides a CPU fallback, and rasterizer/gsplat-metal provides Apple Metal support. The build selects one of these runtime backends, so the application architecture is portable but not runtime-pluggable in the dynamic sense.
9. Framework and library dependencies
The project is built with CMake and links against OpenCV and libtorch. LibTorch supplies tensor storage and autograd-style operations, while OpenCV appears to support image loading and computer vision helpers. That means the training loop is C++/Torch-centric rather than Python-based.
10. Command-line product boundary
The main executable behavior described in the README is: point the program at an existing COLMAP/OpenSfM/OpenMVG/nerfstudio/ODM reconstruction, optionally choose output/compression/resume flags, and emit trained splat assets. There is no requirement for a server, database, or web frontend; this is an offline compute tool with file-based inputs and outputs.
11. Cross-platform and GPU build matrix
The .github/workflows/*, multiple Dockerfile* variants, and README build sections show the repo is explicitly engineered for Linux, Windows, macOS, CUDA, HIP/ROCm, and CPU-only builds. Architecture-wise, this means platform differences are mostly isolated to build configuration and backend kernels rather than the high-level training pipeline.
12. Diagram-worthy data flow
A useful architecture diagram should show external reconstruction data entering through format-specific importers, being normalized into internal scene/model state, then flowing through training and projection/rasterization kernels, with losses/schedulers updating the gaussian model iteratively. The final outputs are serialized splat assets and camera metadata for downstream viewers/editors.
Main entry points: simple_trainer.cpp Trainer · README.md Pipeline · input_data.cpp Input data · project_gaussians.cpp Projection
Components 24
Input adapters
Normalize external SfM/reconstruction formats into the internal scene representation.
- COLMAP · Importer for COLMAP-style reconstructions.
colmap.cpp - OpenMVG · Importer for OpenMVG reconstructions.
openmvg.cpp - OpenSfM · Importer for OpenSfM reconstructions.
opensfm.cpp - Nerfstudio · Importer for Nerfstudio datasets/reconstructions.
nerfstudio.cpp - OpenSplat IO · Native scene format adapter and serialization boundary.
opensplat.cpp
Core model
Scene, camera, point, and export data structures used across the pipeline.
- Input data · Normalized cameras, images, and sparse reconstruction data.
input_data.cpp - Gaussian model · Trainable gaussian scene representation and parameters.
model.cpp - Point I/O · Point-cloud and splat scene read/write bridge.
point_io.cpp - Constants · Shared numeric and structural constants.
constants.hpp
Training loop
Optimization, losses, and progress control for fitting gaussians to images.
- Trainer · Main iterative optimization loop and orchestration.
simple_trainer.cpp - Scheduler · Learning-rate and training schedule control.
optim_scheduler.cpp - SSIM · Image-space structural similarity loss component.
ssim.cpp - Visualizer · Progress display and training-time inspection.
visualizer.cpp
Geometry/math
Projection, tensor, spatial indexing, and shared computation helpers.
- Tensor math · Shared tensor operations and numeric helpers.
tensor_math.cpp - KD-tree · Spatial acceleration structure for neighborhoods and selection.
kdtree_tensor.cpp - Projection · Projects gaussians into image space and tile bounds.
project_gaussians.cpp - Utilities · Common helpers used across training and rendering.
utils.cpp - Spherical harmonics · View-dependent appearance representation.
spherical_harmonics.cpp
Rasterization
Backend-agnostic rasterization entrypoints and backend-specific implementations.
- Rasterizer API · Backend-agnostic entrypoint for gaussian rasterization.
rasterize_gaussians.cpp - CUDA backend · CUDA forward/backward kernels and bindings.
rasterizer/gsplat/forward.cu - CPU backend · CPU rasterization implementation.
rasterizer/gsplat-cpu/gsplat_cpu.cpp - Metal backend · Apple Metal rasterization implementation.
rasterizer/gsplat-metal/gsplat_metal.mm
Build targets
Build system and platform-specific automation for supported runtimes.
- Pipeline · Offline workflow from reconstruction input to trained splat outputs and camera metadata.
README.md - CMake build · Top-level build configuration and dependency wiring.
CMakeLists.txt
Connections 31
- Pipeline → COLMAP: inputs · External reconstructions enter through format-specific importers.
- Pipeline → OpenMVG: inputs · External reconstructions enter through format-specific importers.
- Pipeline → OpenSfM: inputs · External reconstructions enter through format-specific importers.
- Pipeline → Nerfstudio: inputs · External reconstructions enter through format-specific importers.
- Pipeline → OpenSplat IO: outputs · Trained splat assets and camera metadata are emitted for viewers/editors.
- COLMAP → Input data: normalize · Importer converts source format into shared scene data.
- OpenMVG → Input data: normalize · Importer converts source format into shared scene data.
- OpenSfM → Input data: normalize · Importer converts source format into shared scene data.
- Nerfstudio → Input data: normalize · Importer converts source format into shared scene data.
- OpenSplat IO → Point I/O: serialize · Native format is written and read through point/splat I/O.
- Input data → Gaussian model: init · Scene state seeds gaussian model initialization.
- Point I/O → Gaussian model: load/save · Model parameters and splat data pass through point I/O.
- Gaussian model → Trainer: optimize · Trainer updates the gaussian scene model iteratively.
- Scheduler → Trainer: control · Scheduling steers optimization dynamics.
- SSIM → Trainer: loss · Image-space objective contributes to training feedback.
- Visualizer → Trainer: observe · Training progress is reported and inspected.
- Trainer → Projection: forward pass · Training projects gaussians into camera views.
- Trainer → Rasterizer API: render · Forward rendering feeds reconstruction loss computation.
- Projection → Tensor math: math · Projection uses shared numeric and tensor operations.
- Projection → KD-tree: spatial queries · Spatial indexing supports neighborhood and visibility computations.
- Projection → Constants: params · Projection math relies on shared constants.
- Trainer → Spherical harmonics: appearance · View-dependent color modeling uses spherical harmonics.
- Rasterizer API → CUDA backend: backend · CUDA implementation provides the fast path.
- Rasterizer API → CPU backend: backend · CPU implementation provides a fallback path.
- Rasterizer API → Metal backend: backend · Metal implementation provides Apple platform support.
- CUDA backend → Tensor math: compute · Backend kernels depend on shared tensor/math helpers.
- CPU backend → Tensor math: compute · Backend code reuses shared tensor/math helpers.
- Metal backend → Tensor math: compute · Backend code reuses shared tensor/math helpers.
- CMake build → CUDA backend: selects · Build configuration chooses the active backend.
- CMake build → CPU backend: selects · Build configuration chooses the active backend.
- CMake build → Metal backend: selects · Build configuration chooses the active backend.