# jumpat/seganygaussians architecture

> Architecture diagram of the GitHub repository jumpat/seganygaussians, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jumpat/seganygaussians
- Repository: https://github.com/jumpat/seganygaussians

## Overview

SAGA is a research application for segmenting objects represented by 3D Gaussian Splatting. Its documented workflow trains a scene, prepares SAM masks and scale data (and optionally CLIP features), trains Gaussian affinity features, then accepts interactive or open-vocabulary prompts to segment or cluster the scene and render results. The graph preserves these capabilities; links for unsampled scripts and GUI internals rely on README descriptions rather than inspected implementation.

Main entry points: [`train_scene.py`](https://github.com/jumpat/seganygaussians/blob/v2/train_scene.py) (Train scene), [`train_contrastive_feature.py`](https://github.com/jumpat/seganygaussians/blob/v2/train_contrastive_feature.py) (Affinity feature training), [`render.py`](https://github.com/jumpat/seganygaussians/blob/v2/render.py) (Render outputs), [`gaussian_renderer/__init__.py`](https://github.com/jumpat/seganygaussians/blob/v2/gaussian_renderer/__init__.py) (Gaussian renderer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_scene["Scene modeling"]
  node_scene_train["Train scene<br/>[train_scene.py]"]
  node_dataset["Dataset loading<br/>[dataset_readers.py]"]
  node_camera["Camera models<br/>[cameras.py]"]
  node_gaussians["Scene Gaussians<br/>[gaussian_model.py]"]
  node_feature_gaussians["Feature Gaussians"]
  node_scene_state["Scene lifecycle"]
  node_affinity_data["Affinity features"]
end

subgraph group_prep["Feature preparation"]
  node_mask_extract["SAM mask extraction"]
  node_sam_tools["SAM utilities<br/>[sam_utils.py]"]
  node_scale_extract["Mask scale extraction<br/>[get_scale.py]"]
  node_clip_extract["CLIP feature extraction"]
  node_clip_tools["CLIP utilities<br/>[clip_utils.py]"]
  node_affinity_train["Affinity feature training"]
  node_mask_data["SAM masks and scales"]
  node_clip_data["CLIP features"]
end

subgraph group_segment["Segmentation experience"]
  node_gui["Interactive segmentation<br/>[saga_gui.py]"]
  node_open_vocab["Open-vocabulary prompts<br/>[__init__.py]"]
  node_segmentation["Gaussian mask"]
end

subgraph group_render["Rendering and utilities"]
  node_renderer["Gaussian renderer<br/>[__init__.py]"]
  node_rasterizer["CUDA rasterization<br/>[__init__.py]"]
  node_render_cli["Render outputs<br/>[render.py]"]
  node_network_gui["Training viewer<br/>[network_gui.py]"]
  node_geometry["Geometry utilities<br/>[graphics_utils.py]"]
  node_training_utils["Training utilities<br/>[general_utils.py]"]
  node_perceptual_net["Perceptual networks<br/>[networks.py]"]
  node_outputs["Rendered results"]
end

node_user(("Researcher"))

node_user -->|"trains scene"| node_scene_train
node_scene_train -.->|"loads data"| node_dataset
node_scene_train -.->|"optimizes"| node_gaussians
node_scene_train -.->|"saves model"| node_scene_state
node_user -->|"prepares masks"| node_mask_extract
node_mask_extract -.->|"uses SAM"| node_sam_tools
node_mask_extract -->|"writes masks"| node_mask_data
node_user -->|"extracts scales"| node_scale_extract
node_scale_extract -->|"writes scales"| node_mask_data
node_user -->|"extracts features"| node_clip_extract
node_clip_extract -.->|"uses CLIP"| node_clip_tools
node_clip_extract -->|"writes features"| node_clip_data
node_user -->|"trains affinities"| node_affinity_train
node_affinity_train -.->|"loads model"| node_scene_state
node_affinity_train -.->|"trains features"| node_feature_gaussians
node_affinity_train -->|"renders training views"| node_renderer
node_affinity_train -.->|"saves features"| node_affinity_data
node_user -->|"prompts segmentation"| node_gui
node_gui -.->|"loads scene"| node_scene_state
node_gui -->|"renders view"| node_renderer
node_gui -->|"saves mask"| node_segmentation
node_user -->|"queries objects"| node_open_vocab
node_open_vocab -.->|"uses features"| node_clip_data
node_open_vocab -.->|"produces mask"| node_segmentation
node_user -->|"renders scene"| node_render_cli
node_render_cli -.->|"loads model"| node_scene_state
node_render_cli -->|"loads mask"| node_segmentation
node_render_cli -.->|"requests render"| node_renderer
node_renderer -->|"rasterizes Gaussians"| node_rasterizer
node_renderer -->|"reads view"| node_camera
node_render_cli -->|"writes renders"| node_outputs
node_scene_train -->|"serves preview"| node_network_gui
node_camera -->|"builds transforms"| node_geometry
node_gaussians -->|"uses optimization helpers"| node_training_utils

click node_scene_train "https://github.com/jumpat/seganygaussians/blob/v2/train_scene.py"
click node_dataset "https://github.com/jumpat/seganygaussians/blob/v2/scene/dataset_readers.py"
click node_camera "https://github.com/jumpat/seganygaussians/blob/v2/scene/cameras.py"
click node_gaussians "https://github.com/jumpat/seganygaussians/blob/v2/scene/gaussian_model.py"
click node_feature_gaussians "https://github.com/jumpat/seganygaussians/blob/v2/scene/gaussian_model_ff.py"
click node_scene_state "https://github.com/jumpat/seganygaussians/tree/v2/scene"
click node_mask_extract "https://github.com/jumpat/seganygaussians/blob/v2/extract_segment_everything_masks.py"
click node_sam_tools "https://github.com/jumpat/seganygaussians/blob/v2/clip_utils/sam_utils.py"
click node_scale_extract "https://github.com/jumpat/seganygaussians/blob/v2/get_scale.py"
click node_clip_extract "https://github.com/jumpat/seganygaussians/blob/v2/get_clip_features.py"
click node_clip_tools "https://github.com/jumpat/seganygaussians/blob/v2/clip_utils/clip_utils.py"
click node_affinity_train "https://github.com/jumpat/seganygaussians/blob/v2/train_contrastive_feature.py"
click node_gui "https://github.com/jumpat/seganygaussians/blob/v2/saga_gui.py"
click node_open_vocab "https://github.com/jumpat/seganygaussians/blob/v2/clip_utils/__init__.py"
click node_renderer "https://github.com/jumpat/seganygaussians/blob/v2/gaussian_renderer/__init__.py"
click node_rasterizer "https://github.com/jumpat/seganygaussians/blob/v2/submodules/diff-gaussian-rasterization/diff_gaussian_rasterization/__init__.py"
click node_render_cli "https://github.com/jumpat/seganygaussians/blob/v2/render.py"
click node_network_gui "https://github.com/jumpat/seganygaussians/blob/v2/gaussian_renderer/network_gui.py"
click node_geometry "https://github.com/jumpat/seganygaussians/blob/v2/utils/graphics_utils.py"
click node_training_utils "https://github.com/jumpat/seganygaussians/blob/v2/utils/general_utils.py"
click node_perceptual_net "https://github.com/jumpat/seganygaussians/blob/v2/lpipsPyTorch/modules/networks.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scene_train,node_dataset,node_camera,node_gaussians,node_feature_gaussians,node_scene_state,node_affinity_data toneBlue
class node_mask_extract,node_sam_tools,node_scale_extract,node_clip_extract,node_clip_tools,node_affinity_train,node_mask_data,node_clip_data toneAmber
class node_gui,node_open_vocab,node_segmentation toneMint
class node_renderer,node_rasterizer,node_render_cli,node_network_gui,node_geometry,node_training_utils,node_perceptual_net,node_outputs toneRose
class node_user toneIndigo
```

## Components

### Scene modeling

- [Train scene](https://github.com/jumpat/seganygaussians/blob/v2/train_scene.py): `train_scene.py`
- [Dataset loading](https://github.com/jumpat/seganygaussians/blob/v2/scene/dataset_readers.py): `scene/dataset_readers.py`
- [Camera models](https://github.com/jumpat/seganygaussians/blob/v2/scene/cameras.py): `scene/cameras.py`
- [Scene Gaussians](https://github.com/jumpat/seganygaussians/blob/v2/scene/gaussian_model.py): `scene/gaussian_model.py`
- [Feature Gaussians](https://github.com/jumpat/seganygaussians/blob/v2/scene/gaussian_model_ff.py): `scene/gaussian_model_ff.py`
- [Scene lifecycle](https://github.com/jumpat/seganygaussians/tree/v2/scene): `scene`
- **Affinity features**

### Feature preparation

- [SAM mask extraction](https://github.com/jumpat/seganygaussians/blob/v2/extract_segment_everything_masks.py): `extract_segment_everything_masks.py`
- [SAM utilities](https://github.com/jumpat/seganygaussians/blob/v2/clip_utils/sam_utils.py): `clip_utils/sam_utils.py`
- [Mask scale extraction](https://github.com/jumpat/seganygaussians/blob/v2/get_scale.py): `get_scale.py`
- [CLIP feature extraction](https://github.com/jumpat/seganygaussians/blob/v2/get_clip_features.py): `get_clip_features.py`
- [CLIP utilities](https://github.com/jumpat/seganygaussians/blob/v2/clip_utils/clip_utils.py): `clip_utils/clip_utils.py`
- [Affinity feature training](https://github.com/jumpat/seganygaussians/blob/v2/train_contrastive_feature.py): `train_contrastive_feature.py`
- **SAM masks and scales**
- **CLIP features**

### Segmentation experience

- [Interactive segmentation](https://github.com/jumpat/seganygaussians/blob/v2/saga_gui.py): `saga_gui.py`
- [Open-vocabulary prompts](https://github.com/jumpat/seganygaussians/blob/v2/clip_utils/__init__.py): `clip_utils/__init__.py`
- **Gaussian mask**

### Rendering and utilities

- [Gaussian renderer](https://github.com/jumpat/seganygaussians/blob/v2/gaussian_renderer/__init__.py): `gaussian_renderer/__init__.py`
- [CUDA rasterization](https://github.com/jumpat/seganygaussians/blob/v2/submodules/diff-gaussian-rasterization/diff_gaussian_rasterization/__init__.py): `submodules/diff-gaussian-rasterization/diff_gaussian_rasterization/__init__.py`
- [Render outputs](https://github.com/jumpat/seganygaussians/blob/v2/render.py): `render.py`
- [Training viewer](https://github.com/jumpat/seganygaussians/blob/v2/gaussian_renderer/network_gui.py): `gaussian_renderer/network_gui.py`
- [Geometry utilities](https://github.com/jumpat/seganygaussians/blob/v2/utils/graphics_utils.py): `utils/graphics_utils.py`
- [Training utilities](https://github.com/jumpat/seganygaussians/blob/v2/utils/general_utils.py): `utils/general_utils.py`
- [Perceptual networks](https://github.com/jumpat/seganygaussians/blob/v2/lpipsPyTorch/modules/networks.py): `lpipsPyTorch/modules/networks.py`
- **Rendered results**

### Other components

- **Researcher**

## Connections

- Researcher → Train scene: trains scene
- Train scene → Dataset loading: loads data
- Train scene → Scene Gaussians: optimizes
- Train scene → Scene lifecycle: saves model (evidence: `train_scene.py`)
- Researcher → SAM mask extraction: prepares masks
- SAM mask extraction → SAM utilities: uses SAM
- SAM mask extraction → SAM masks and scales: writes masks
- Researcher → Mask scale extraction: extracts scales
- Mask scale extraction → SAM masks and scales: writes scales
- Researcher → CLIP feature extraction: extracts features
- CLIP feature extraction → CLIP utilities: uses CLIP
- CLIP feature extraction → CLIP features: writes features
- Researcher → Affinity feature training: trains affinities
- Affinity feature training → Scene lifecycle: loads model (evidence: `train_contrastive_feature.py`)
- Affinity feature training → Feature Gaussians: trains features
- Affinity feature training → Gaussian renderer: renders training views (evidence: `train_contrastive_feature.py`)
- Affinity feature training → Affinity features: saves features
- Researcher → Interactive segmentation: prompts segmentation
- Interactive segmentation → Scene lifecycle: loads scene (evidence: `saga_gui.py`)
- Interactive segmentation → Gaussian renderer: renders view (evidence: `saga_gui.py`)
- Interactive segmentation → Gaussian mask: saves mask
- Researcher → Open-vocabulary prompts: queries objects
- Open-vocabulary prompts → CLIP features: uses features
- Open-vocabulary prompts → Gaussian mask: produces mask
- Researcher → Render outputs: renders scene
- Render outputs → Scene lifecycle: loads model
- Render outputs → Gaussian mask: loads mask
- Render outputs → Gaussian renderer: requests render
- Gaussian renderer → CUDA rasterization: rasterizes Gaussians (evidence: `gaussian_renderer/__init__.py`)
- Gaussian renderer → Camera models: reads view (evidence: `gaussian_renderer/__init__.py`)
- Render outputs → Rendered results: writes renders
- Train scene → Training viewer: serves preview (evidence: `train_scene.py`)
- Camera models → Geometry utilities: builds transforms (evidence: `scene/cameras.py`)
- Scene Gaussians → Training utilities: uses optimization helpers (evidence: `scene/gaussian_model.py`)