# borglab/gtsfm architecture

> Architecture diagram of the GitHub repository borglab/gtsfm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/borglab/gtsfm
- Repository: https://github.com/borglab/gtsfm

## Overview

##### High-Level Components

a) Front-End Layer
- Feature Detection/Description Module
- Multiple implementations (SIFT, SuperPoint, etc.)
- To Detector and Descriptor submodules
- Matching Module
- SuperGlue, traditional matchers
- Verification Module
- Degensac, RANSAC implementations
- Caching System
- Detector/descriptor and matcher caching

b) Middle Layer
- Data Association Module
- Connection to 3D point triangulation
- Averaging Module
- Rotation Averaging (Shonan)
- Translation Averaging (1D-SFM)

c) Back-End Layer
- Bundle Adjustment Module
- Dense Reconstruction Module (PatchMatchNet)

##### Data Flow Connections
- Image data flow from Loaders through the pipeline
- Feature extraction → matching → verification flow
- Point tracks formation
- Camera pose estimation flow
- Final 3D reconstruction output

##### Distributed Computing Layer
Show:
- Dask integration for parallel computation
- Worker distribution
- Task scheduling

##### Supporting Systems
Include:
- Data Loaders (various formats)
- Visualization Tools
- Evaluation Metrics
- Caching System

##### External Integrations
Connections to:
- GTSAM
- Nerfstudio
- COLMAP (for comparisons)

Use the following visual elements:
- Rounded rectangles for submodules
- Blue: Front-end components
- Green: Middle-layer components
- Orange: Back-end components
- Gray: Supporting infrastructure
- Purple: External integrations

Key Design Principles to Highlight:
##### Integration Points: Display external tool connections

- Component types
- Implementation options for each module

This design captures both the high-level architecture and the detailed module interactions while highlighting the system's key features: distributed computation, modular design, and extensive front-end capabilities.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Legend
        L1[Module]
        L2([Submodule])
        L3[(Database)]
        L4>External Integration]
        style L1 fill:#a8d1f0
        style L2 fill:#b8e0b8
        style L3 fill:#f5d6a7
        style L4 fill:#e6b3e6
    end

    subgraph "Front-End Layer"
        FD["Feature Detection/Description"]:::frontend
        MM["Matching Module"]:::frontend
        VM["Verification Module"]:::frontend
        CS["Caching System"]:::frontend

        subgraph "Feature Detection Options"
            SIFT([SIFT]):::submodule
            SP([SuperPoint]):::submodule
        end

        subgraph "Matching Options"
            SG([SuperGlue]):::submodule
            LG([LightGlue]):::submodule
            TM([Traditional Matcher]):::submodule
        end

        subgraph "Verification Options"
            RANSAC([RANSAC]):::submodule
            DEGEN([Degensac]):::submodule
        end

        subgraph "Cache Components"
            DDC([Detector/Descriptor Cache]):::submodule
            MC([Matcher Cache]):::submodule
        end
    end

    subgraph "Middle Layer"
        DA["Data Association"]:::middle
        AM["Averaging Module"]:::middle

        subgraph "Averaging Components"
            RA([Rotation Averaging - Shonan]):::submodule
            TA([Translation Averaging - 1DSFM]):::submodule
        end
    end

    subgraph "Back-End Layer"
        BA["Bundle Adjustment"]:::backend
        DR["Dense Reconstruction"]:::backend
        PMN([PatchMatchNet]):::submodule
    end

    subgraph "Supporting Systems"
        DL["Data Loaders"]:::support
        VT["Visualization Tools"]:::support
        EM["Evaluation Metrics"]:::support
    end

    subgraph "External Integrations"
        GTSAM>GTSAM]:::external
        NS>Nerfstudio]:::external
        COLMAP>COLMAP]:::external
    end

    subgraph "Distributed Computing"
        DASK[(Dask)]:::database
    end

    DL --> FD
    FD --> |Features| MM
    MM --> |Matches| VM
    VM --> |Verified Matches| DA
    DA --> AM
    AM --> BA
    BA --> DR
    DR --> PMN

    FD -.-> DDC
    MM -.-> MC
    
    BA --> GTSAM
    DR --> NS
    EM --> COLMAP

    DASK -.- FD
    DASK -.- MM
    DASK -.- VM
    DASK -.- BA

    classDef frontend fill:#a8d1f0
    classDef middle fill:#90EE90
    classDef backend fill:#FFB347
    classDef support fill:#D3D3D3
    classDef external fill:#DDA0DD
    classDef submodule fill:#b8e0b8
    classDef database fill:#f5d6a7

    click FD "https://github.com/borglab/gtsfm/tree/master/gtsfm/frontend/detector/"
    click MM "https://github.com/borglab/gtsfm/tree/master/gtsfm/frontend/matcher/"
    click VM "https://github.com/borglab/gtsfm/tree/master/gtsfm/frontend/verifier/"
    click CS "https://github.com/borglab/gtsfm/tree/master/gtsfm/frontend/cacher/"
    click DA "https://github.com/borglab/gtsfm/tree/master/gtsfm/data_association/"
    click AM "https://github.com/borglab/gtsfm/tree/master/gtsfm/averaging/"
    click BA "https://github.com/borglab/gtsfm/tree/master/gtsfm/bundle/"
    click DR "https://github.com/borglab/gtsfm/tree/master/gtsfm/densify/"
    click DL "https://github.com/borglab/gtsfm/tree/master/gtsfm/loader/"
    click VT "https://github.com/borglab/gtsfm/tree/master/gtsfm/visualization/"
    click EM "https://github.com/borglab/gtsfm/tree/master/gtsfm/evaluation/"
    click SG "https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/matcher/superglue_matcher.py"
    click LG "https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/matcher/lightglue_matcher.py"
    click RANSAC "https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/verifier/ransac.py"
    click DEGEN "https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/verifier/degensac.py"
    click DDC "https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/cacher/detector_descriptor_cacher.py"
    click MC "https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/cacher/matcher_cacher.py"
    click RA "https://github.com/borglab/gtsfm/blob/master/gtsfm/averaging/rotation/shonan.py"
    click TA "https://github.com/borglab/gtsfm/blob/master/gtsfm/averaging/translation/averaging_1dsfm.py"
    click PMN "https://github.com/borglab/gtsfm/blob/master/gtsfm/densify/mvs_patchmatchnet.py"
```
