How vast-ai-research/triposplat works
1. What this repo is
TripoSplat is a single-image-to-3D-Gaussian generation pipeline. The architectural center is an inference stack that takes a 2D image, runs a learned model, and exports a variable-sized Gaussian splat representation for downstream rendering or editing.
2. Core code split
The repository is intentionally small and centered on two source files:
- triposplat.py: the main pipeline/orchestration layer.
- model.py: the model definition and related tensor-level logic.
This split suggests a boundary between “system/inference flow” and “network/components.”
3. Primary data flow
The main flow is:
input image → preprocessing/loaders → TripoSplat model inference → generated 3D Gaussian parameters → export to .ply / .splat formats → external viewer or downstream DCC/graphics tool.
4. Checkpoint boundary
Model weights are not stored in the repo; they are downloaded into ckpts/ from Hugging Face or ModelScope. That makes the learned weights an external dependency boundary, separate from the codebase itself.
5. Inference entry points
There are two runnable paths:
- run_example.py for local example/batch inference.
- run_gradio.py for an interactive demo UI.
Both are thin application layers over the same core inference code, so they are not separate model implementations.
6. Export and interoperability
The output is designed to be portable: exported .ply / .splat files can be opened in third-party Gaussian viewers like SparkJS or SuperSplat. This means the repo’s responsibility ends at producing standard splat artifacts, not rendering them itself.
7. Built-in viewer assets
static/viewer/viewer.html indicates an embedded browser-based viewer, likely used for previewing results locally. The static/doc and static/example_inputs directories are supporting assets rather than core logic: documentation imagery and sample inputs for demos/examples.
8. Demo/UI layer
run_gradio.py adds a lightweight human-facing interface around the inference pipeline. Architecturally, this is a presentation layer that accepts an uploaded image, invokes the model, and presents downloadable 3D outputs.
9. Dependency profile
The repo emphasizes minimal runtime dependencies. From the README, the practical stack is PyTorch plus small utility libraries (numpy, safetensors, pillow, tqdm), with gradio only needed for the demo. This matters architecturally because the project is self-contained and avoids larger orchestration frameworks.
10. Repository boundary
There is no separate training code, service backend, or database layer in this repo. It is primarily an inference-and-export package with demo wrappers and static visualization assets, intended to be embedded into pipelines such as ComfyUI or used as a local tool.
Main entry points: triposplat.py TripoSplat · run_gradio.py Gradio demo · run_example.py Example run · model.py Model
Components 16
Core inference
Main image-to-Gaussian generation pipeline and model logic.
- TripoSplat · Main orchestration layer for preprocessing, inference, and export.
triposplat.py - Model · Model definition and tensor-level inference logic.
model.py - Preprocess · Loads and prepares the input image for inference.
triposplat.py - Infer splats · Runs the learned model to produce 3D Gaussian parameters.
model.py - Postprocess · Converts model output into portable splat artifacts.
triposplat.py - PLY export · Writes standard .ply output for downstream tools.
triposplat.py - SPLAT export · Writes standard .splat output for Gaussian viewers.
triposplat.py
Runtimes
Runnable entry points wrapping the core pipeline.
- Example run · Local example and batch inference entry point.
run_example.py - Gradio demo · Interactive demo entry point for uploaded images and downloads.
run_gradio.py
Assets
Static viewer, docs, and sample inputs for demos and preview.
- Viewer · Embedded browser-based preview for generated splats.
static/viewer/viewer.html - Sample inputs · Example images used to demonstrate the pipeline.
static/example_inputs - Docs assets · Static documentation and supporting imagery.
static/doc
External boundary
Model checkpoints and third-party tools outside the repository.
- Input image · Single 2D image used as the source for generation.
- Checkpoints · Downloaded model weights stored outside the repo in ckpts/.
- Third-party viewers · External Gaussian viewers such as SparkJS or SuperSplat.
- Downstream tools · External rendering or editing pipelines that consume exported splats.
Connections 16
- Input image → Preprocess: load · The source image is loaded and prepared for inference.
- Preprocess → TripoSplat: orchestrate · Main pipeline coordinates preprocessing and execution.
- TripoSplat → Model: calls · Pipeline invokes the model implementation.
- Model → Checkpoints: loads · Model parameters are provided by external checkpoints.
- TripoSplat → Infer splats: run · Inference produces 3D Gaussian parameters.
- Infer splats → Postprocess: emit · Model output is prepared for export.
- Postprocess → PLY export: write · Produces .ply artifacts.
- Postprocess → SPLAT export: write · Produces .splat artifacts.
- PLY export → Downstream tools: consume · PLY files can be used by external tools.
- SPLAT export → Third-party viewers: preview · SPLAT files can be opened in third-party viewers.
- TripoSplat → Viewer: preview · The pipeline can surface results through the embedded viewer.
- Example run → TripoSplat: uses · Example runner is a thin wrapper over the core pipeline.
- Example run → Sample inputs: reads · Uses bundled sample inputs for local runs.
- Gradio demo → TripoSplat: uses · Demo UI calls into the same core inference path.
- Gradio demo → Viewer: shows · UI can present local preview output through the embedded viewer.
- Docs assets → Gradio demo: supports · Documentation assets support the demo and repo instructions.