# shashankyld/dynamic_gs architecture

> Architecture diagram of the GitHub repository shashankyld/dynamic_gs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shashankyld/dynamic_gs
- Repository: https://github.com/shashankyld/dynamic_gs

## Overview

##### Overall System “Layers” and Main Components
- Frontend / Data Acquisition:
- Unified DataLoader (see “Frame”, “KeyFrame” concepts in the README).
- Input sensors: RGBD frames (and possibly other lighter modalities).
- Feature Processing and Tracking:
- Feature Detection & Matching: Using Delaunay triangulation–based matching (removing current frame Delaunay each time and tracking via the reference frame).
- Frame-to-frame, frame-to-sparse map, and image-to-GS map tracking.
- Feature Det/Match module that also collects common feature counts to help decide keyframe spawning.
- Dynamic Object Analysis:
- SAM2 integration for proposing dynamic masks.
- Adjustment/improvement of prompts based on consistency with the Delaunay feature matching.
- SLAM Backend:
- Core SLAM module that handles pose estimation, loop closure, and keyframe management.
- Factor graph optimization block (using gtsam/g20/pypose).
- TimingUtils integrated throughout to track each method’s latency.
- Mapping Subsystems:
- Sparse Map module: Maintains list of keyframes and associated metadata (features, tracked pose, dynamic detection results).
- Dense Map (GS Map) module: Manages a list of “Gaussians” (each attached to a keyframe), with operations such as spawn, prune, and densify.
- Parallel Processing and Worker Threads:
- The main thread handles feature detection, pose estimation, and core SLAM operations.
- Worker threads execute SAM2 processing, Gaussian splatting optimization, loop closure detection, and map maintenance.
- Visualization and Testing:
- Real-time trajectory visualization and map quality assessment (both interactive and batch mode).
- Testing frameworks and benchmarking modules (see tests/ and performance target benchmarks in the README).

##### Architectural Relationships and Data Flow
- Data Flow: Start with the DataLoader reading input frames ➔ Each frame is processed by the Feature Detector Matcher module ➔ The extracted features and matches feed into the Track and SLAM modules for pose estimation.
- Dynamic processing: Parallel to tracking, the dynamic mask generation module (powered by SAM2) receives input images and produces masks that are used both for refining prompts and gating the mapping update.
- Mapping: Once a frame qualifies as a keyframe (based on common feature counts and metric thresholds), it is pushed into the Sparse Map where its full set of data (features, pose, dynamic mask) is stored. In parallel, the Dense Map module updates its Gaussian Splatting representation using data from the keyframes.
- Factor Graph & Loop Closure: The SLAM backend sends data to a factor graph optimization module and receives corrections via loop closure detections which are handled in worker threads.
- Visualization: Both the trajectory (from pose estimation) and map quality (sparse and dense maps) are forwarded to visualization modules for real-time display and debugging.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend / Data Acquisition
    subgraph "Frontend / Data Acquisition"
        sensor["RGBD Sensor"]:::acquisition
        dataloader["Unified DataLoader"]:::acquisition
        sensor --> dataloader
    end

    %% Feature Processing and Tracking
    subgraph "Feature Processing and Tracking"
        feat["Feature Detection & Matching (Delaunay)"]:::tracking
        track["Tracking & Drawing"]:::tracking
        dataloader --> feat
        feat -->|"visual_feedback"| track
    end

    %% Dynamic Object Analysis
    subgraph "Dynamic Object Analysis"
        dyn["Dynamic Mask Generator (SAM2)"]:::dynamic
        feat --> dyn
    end

    %% SLAM Backend
    subgraph "SLAM Backend"
        slam["SLAM & Pose Estimation"]:::slam
        loop["Factor Graph Optimization/LoopClosure"]:::slam
        timing["Timing Utilities"]:::slam
        feat --> slam
        dyn -->|"mask_update"| slam
        slam -->|"push_keyframe"| loop
        loop -->|"feedback"| slam
        slam --- timing
    end

    %% Mapping Subsystems
    subgraph "Mapping Subsystems"
        sparse["Sparse Map"]:::mapping
        dense["Dense (GS) Map"]:::mapping
        slam --> sparse
        sparse --> dense
    end

    %% Parallel Processing and Worker Threads
    subgraph "Parallel Processing"
        main["Main Thread"]:::parallel
        worker["Worker Threads"]:::parallel
        slam --> main
        main --> worker
        dyn --> worker
        loop --> worker
    end

    %% Visualization and Testing
    subgraph "Visualization & Testing"
        viztools["Visualization Tools"]:::viz
        tests["Testing Frameworks"]:::viz
        slam --> viztools
        sparse --> viztools
        dense --> viztools
        viztools --> tests
    end

    %% Click Events for Data Acquisition
    click dataloader "https://github.com/shashankyld/dynamic_gs/blob/master/frame.py"
    click dataloader "https://github.com/shashankyld/dynamic_gs/blob/master/io_utils/dataset.py"
    click dataloader "https://github.com/shashankyld/dynamic_gs/blob/master/utilities/utils_data.py"

    %% Click Events for Feature Processing and Tracking
    click feat "https://github.com/shashankyld/dynamic_gs/blob/master/utilities/utils_features.py"
    click feat "https://github.com/shashankyld/dynamic_gs/blob/master/utilities/utils_delaunay.py"
    click track "https://github.com/shashankyld/dynamic_gs/blob/master/utilities/utils_draw.py"

    %% Click Events for Dynamic Object Analysis
    click dyn "https://github.com/shashankyld/dynamic_gs/tree/master/thirdparty/sam2"
    click dyn "https://github.com/shashankyld/dynamic_gs/blob/master/utilities/utils_maskrcnn.py"

    %% Click Events for SLAM Backend
    click slam "https://github.com/shashankyld/dynamic_gs/blob/master/slam.py"
    click slam "https://github.com/shashankyld/dynamic_gs/blob/master/utilities/utils_tf.py"
    click slam "https://github.com/shashankyld/dynamic_gs/blob/master/utilities/utils_rerun.py"
    click slam "https://github.com/shashankyld/dynamic_gs/blob/master/utilities/timer.py"

    %% Click Event for Mapping Subsystems
    click sparse "https://github.com/shashankyld/dynamic_gs/blob/master/io_utils/trajectory_writer.py"
    click dense "https://github.com/shashankyld/dynamic_gs/tree/master/thirdparty/gaussian_splatting/"

    %% Click Event for Parallel Processing
    click worker "https://github.com/shashankyld/dynamic_gs/blob/master/utilities/utils_mp.py"

    %% Click Events for Visualization and Testing
    click viztools "https://github.com/shashankyld/dynamic_gs/blob/master/visualization_utils.py"
    click viztools "https://github.com/shashankyld/dynamic_gs/tree/master/visualizations/"
    click tests "https://github.com/shashankyld/dynamic_gs/tree/master/tests/"

    %% Styles
    classDef acquisition fill:#f4d9f9,stroke:#333,stroke-width:2px;
    classDef tracking fill:#cce5ff,stroke:#333,stroke-width:2px;
    classDef dynamic fill:#d4f7d4,stroke:#333,stroke-width:2px;
    classDef slam fill:#ffe5cc,stroke:#333,stroke-width:2px;
    classDef mapping fill:#fff4cc,stroke:#333,stroke-width:2px;
    classDef parallel fill:#d1eefc,stroke:#333,stroke-width:2px;
    classDef viz fill:#ffd6d6,stroke:#333,stroke-width:2px;
```
