How jumpat/seganygaussians works
SAGA is a research application for segmenting objects represented by 3D Gaussian Splatting. Its documented workflow trains a scene, prepares SAM masks and scale data (and optionally CLIP features), trains Gaussian affinity features, then accepts interactive or open-vocabulary prompts to segment or cluster the scene and render results. The graph preserves these capabilities; links for unsampled scripts and GUI internals rely on README descriptions rather than inspected implementation.
Main entry points: train_scene.py Train scene · train_contrastive_feature.py Affinity feature training · render.py Render outputs · __init__.py Gaussian renderer
Components 27
Scene modeling
- Train scene
train_scene.py - Dataset loading
scene/dataset_readers.py - Camera models
scene/cameras.py - Scene Gaussians
scene/gaussian_model.py - Feature Gaussians
scene/gaussian_model_ff.py - Scene lifecycle
scene - Affinity features
Feature preparation
- SAM mask extraction
extract_segment_everything_masks.py - SAM utilities
clip_utils/sam_utils.py - Mask scale extraction
get_scale.py - CLIP feature extraction
get_clip_features.py - CLIP utilities
clip_utils/clip_utils.py - Affinity feature training
train_contrastive_feature.py - SAM masks and scales
- CLIP features
Segmentation experience
- Interactive segmentation
saga_gui.py - Open-vocabulary prompts
clip_utils/__init__.py - Gaussian mask
Rendering and utilities
- Gaussian renderer
gaussian_renderer/__init__.py - CUDA rasterization
submodules/diff-gaussian-rasterization/diff_gaussian_rasterization/__init__.py - Render outputs
render.py - Training viewer
gaussian_renderer/network_gui.py - Geometry utilities
utils/graphics_utils.py - Training utilities
utils/general_utils.py - Perceptual networks
lpipsPyTorch/modules/networks.py - Rendered results
Other components
- Researcher
Connections 34
- Researcher → Train scene: trains scene
- Train scene → Dataset loading: loads data
- Train scene → Scene Gaussians: optimizes
- Train scene → Scene lifecycle: saves model
train_scene.py - Researcher → SAM mask extraction: prepares masks
- SAM mask extraction → SAM utilities: uses SAM
- SAM mask extraction → SAM masks and scales: writes masks
- Researcher → Mask scale extraction: extracts scales
- Mask scale extraction → SAM masks and scales: writes scales
- Researcher → CLIP feature extraction: extracts features
- CLIP feature extraction → CLIP utilities: uses CLIP
- CLIP feature extraction → CLIP features: writes features
- Researcher → Affinity feature training: trains affinities
- Affinity feature training → Scene lifecycle: loads model
train_contrastive_feature.py - Affinity feature training → Feature Gaussians: trains features
- Affinity feature training → Gaussian renderer: renders training views
train_contrastive_feature.py - Affinity feature training → Affinity features: saves features
- Researcher → Interactive segmentation: prompts segmentation
- Interactive segmentation → Scene lifecycle: loads scene
saga_gui.py - Interactive segmentation → Gaussian renderer: renders view
saga_gui.py - Interactive segmentation → Gaussian mask: saves mask
- Researcher → Open-vocabulary prompts: queries objects
- Open-vocabulary prompts → CLIP features: uses features
- Open-vocabulary prompts → Gaussian mask: produces mask
- Researcher → Render outputs: renders scene
- Render outputs → Scene lifecycle: loads model
- Render outputs → Gaussian mask: loads mask
- Render outputs → Gaussian renderer: requests render
- Gaussian renderer → CUDA rasterization: rasterizes Gaussians
gaussian_renderer/__init__.py - Gaussian renderer → Camera models: reads view
gaussian_renderer/__init__.py - Render outputs → Rendered results: writes renders
- Train scene → Training viewer: serves preview
train_scene.py - Camera models → Geometry utilities: builds transforms
scene/cameras.py - Scene Gaussians → Training utilities: uses optimization helpers
scene/gaussian_model.py