How meshsplatting/mesh-splatting works
MeshSplatting trains differentiable opaque-triangle scene representations from camera images, then supports rendering, evaluation, mesh export, and object extraction. The main workflow loads COLMAP or synthetic-scene data, optimizes a triangle model through differentiable rendering, and exports or evaluates the result. Optional depth and normal supervision and the documented SAM-based extraction flow are shown as separate branches. Some entry-point implementations were not sampled; paths and links for those capabilities rely on the README and file tree, so uncertain internal wiring is omitted.
Main entry points: triangle_renderer/__init__.py Differentiable renderer · scene/__init__.py Scene loader · train.py Training loop · dataset_readers.py Dataset readers
Components 25
Scene data
- Scene loader
scene/__init__.py - Dataset readers
scene/dataset_readers.py - COLMAP model IO
scene/colmap_loader.py - Camera model
scene/cameras.py - Scene images
Model training
- Training loop
train.py - Triangle model
scene/triangle_model.py - Differentiable renderer
triangle_renderer/__init__.py - Training losses
utils/loss_utils.py
Render and evaluate
- Scene rendering
render.py - Video creation
create_video.py - Evaluation
full_eval.py - Image metrics
metrics.py - Scene PLY export
create_ply.py
Object extraction
- Mask annotation
annotate_points_boxes.py - View extraction
segmentation/extract_images.py - SAM mask generation
segmentation/sam_mask_generator_json.py - Triangle assignment
segmentation/segment.py - Object rendering
segmentation/run_single_object.py - Object PLY export
segmentation/create_ply.py
Optional supervision
- Normal extraction
extract_normals.py - Depth scale prep
utils/make_depth_scale.py
Other components
- Researcher
- Depth estimator
- SAM model
Connections 40
- Researcher → Training loop: starts training
- Training loop → Scene loader: loads scene
- Scene loader → Dataset readers: selects reader
- Dataset readers → COLMAP model IO: reads reconstruction
- Dataset readers → Scene images: loads images
- Dataset readers → Camera model: creates cameras
- Scene images → Dataset readers: provides images
- Training loop → Triangle model: optimizes model
- Training loop → Differentiable renderer: renders views
- Training loop → Training losses: computes losses
- Differentiable renderer → Triangle model: reads geometry
- Differentiable renderer → Camera model: uses camera
- Normal extraction → Scene images: writes normals
- Depth estimator → Scene images: produces depth maps
- Depth scale prep → Scene images: writes depth parameters
- Researcher → Scene rendering: requests render
- Scene rendering → Scene loader: loads scene
- Scene rendering → Differentiable renderer: renders views
- Researcher → Video creation: requests video
- Video creation → Scene loader: loads scene
- Video creation → Differentiable renderer: renders frames
- Researcher → Evaluation: requests evaluation
- Evaluation → Scene rendering: runs rendering
- Evaluation → Image metrics: computes metrics
- Researcher → Scene PLY export: requests export
- Scene PLY export → Triangle model: loads trained model
- Researcher → Mask annotation: marks object
- Researcher → View extraction: starts extraction
- View extraction → Scene loader: loads scene
- View extraction → Differentiable renderer: renders views
- Researcher → SAM mask generation: requests masks
- SAM mask generation → SAM model: generates masks
- Researcher → Triangle assignment: assigns triangles
- Triangle assignment → Scene loader: loads scene
- Triangle assignment → Differentiable renderer: renders views
- Triangle assignment → Triangle model: reads triangles
- Researcher → Object rendering: renders selection
- Object rendering → Scene loader: loads scene
- Object rendering → Differentiable renderer: renders selected triangles
- Researcher → Object PLY export: exports selection