# luohongk/supervins architecture

> Architecture diagram of the GitHub repository luohongk/supervins, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/luohongk/supervins
- Repository: https://github.com/luohongk/supervins

## Overview

##### Project type and purpose
- This is a robotics middleware application: a real-time visual-inertial SLAM (Simultaneous Localization and Mapping) framework built on ROS1 Noetic.
- It fuses camera images and IMU measurements, uses deep-learning feature detectors/descriptors (SuperPoint + LightGlue), and performs loop-closure detection (DBoW3 + SuperPoint) and pose graph optimization.

##### File structure
- Top-level modules:
- camera_models: intrinsic calibration, distortion models
- supervins_estimator: the VINS front-end/back-end estimator (feature tracking, initialization, sliding-window optimization)
- supervins_loop_fusion: loop-closure detection & global pose-graph optimization
- config: YAML files for datasets & ROS RViz
- resources/weights_dpl: ONNX models for SuperPoint, LightGlue
- support_files: descriptors & patterns for BRIEF
- Build & deployment: ROS launch files, CMakeLists, package.xml for ROS packages.
- Third-party deps: Ceres, Eigen, OpenCV, ONNX Runtime, DBoW3, ROS Noetic.

##### README insights
- Two ROS nodes: supervins_node and supervins_loop_fusion_node, launched via supervins_rviz.launch.
- Workflow: download data → configure paths → catkin_make → roslaunch + rosrun → rosbag play.

##### Main components & interactions
a. Sensor Inputs
- ROS topics: /camera/image_raw → Image messages
/imu/data → IMU messages
b. Front-End Feature Processing (supervins_estimator)
- SuperPoint ONNX model (ONNX Runtime) → keypoint detection & descriptors
- LightGlue ONNX model → feature matching across consecutive frames
- FeatureTracker module
c. Initialization
- Solve_5pts, initial_sfm, initial_alignment → bootstraps monocular SLAM
d. Back-End Estimation
- Factor graph built with Ceres Solver: IMU factors, projection factors, pose/bias priors
- Sliding-window optimization → state (poses, velocities, biases, landmarks)
e. Loop Closure (supervins_loop_fusion)
- Keyframe selection → extract SuperPoint descriptors
- Build/Query DBoW3 database → detect loop candidates
- Pose-graph construction & optimization → global consistency
f. Outputs & Visualization
- Publish optimized poses → ROS topic (e.g., /vins/odometry)
- RViz visualization via supervins_rviz.launch
- Log files: vio_output.txt, pose_graph_output.txt

##### Architectural patterns & design principles
- Modular ROS nodes: separation of concerns (estimation vs. loop closure).
- Pipeline pattern: sensor acquisition → front-end → initialization → back-end → output.
- Factor-graph optimization (sliding window) + pose-graph optimization (loop fusion).
- Plugin-style camera models (factory + inheritance), facilitating extensibility for new camera types.

- supervins_node (Estimation)
- supervins_loop_fusion_node (Loop Closure)
2. Within supervins_node, partition into sub-blocks:
- Image & IMU Subscribers
- SuperPoint Detector (ONNX Runtime)
- LightGlue Matcher (ONNX Runtime)
- FeatureTracker
- Initializer (solve_5pts, initial_sfm, etc.)
- Ceres Optimizer (factor graph)
3. Within loop_fusion node:
- Keyframe Manager
- SuperPoint Extractor
- DBoW3 Database
- Loop Detector
- Pose-graph Optimizer (Third-party libs)
4. A camera_models module as a shared library used by both nodes (intrinsics, distortion).
5. External dependencies:
- Ceres, Eigen, OpenCV, ONNX Runtime, ROS, DBoW3
- /camera/image_raw → SuperPoint Detector
- /imu/data → IMU factor builder
- Optimized poses → /vins/odometry → RViz
- Keyframe data → loop_fusion node
- Blue: ROS nodes & topics
- Green: Processing modules (feature detection, matching, tracking)
- Orange: Optimization engines (Ceres, Pose Graph)
- Gray: External libs & resources (ONNX models, camera_models)

##### Important notes
- Essential modules and their data flows—do not omit the ONNX feature pipelines or factor graph stages.
- How camera_models is injected into projection factors and feature back-projection.
- How loop closure feeds back into the global state (pose graph).
- Be granular: separate initialization, tracking, and optimization as distinct sub-components.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% ROS Nodes
    subgraph "supervins_node (Estimation)"
        direction TB
        Entry["ROS Node Main\nsupervins_main.cpp"]:::ros
        ImgSub["/camera/image_raw\nSubscriber"]:::ros
        IMUSub["/imu/data\nSubscriber"]:::ros
        SPDetector["SuperPoint Detector\n(ONNX Runtime)"]:::processing
        LGMatcher["LightGlue Matcher\n(ONNX Runtime)"]:::processing
        FeatureTracker["FeatureTracker Module"]:::processing
        Initializer["Initializer\n(solve_5pts, initial_sfm)"]:::processing
        CeresOptimizer["Ceres Optimizer\n(factor graph)"]:::optimization
        UtilityEst["Utility & Visualization Helpers"]:::processing
        LaunchRViz["RViz Launch\nsupervins_rviz.launch"]:::ros
        VIOLogs["VIO Output Logs\nvio_output.txt"]:::ros
        Entry --> ImgSub
        Entry --> IMUSub
        ImgSub --> SPDetector
        SPDetector --> LGMatcher
        LGMatcher --> FeatureTracker
        FeatureTracker --> Initializer
        IMUSub --> CeresOptimizer
        Initializer --> CeresOptimizer
        CeresOptimizer -->|"/vins/odometry"| LaunchRViz
        CeresOptimizer --> VIOLogs
        UtilityEst --> LaunchRViz
    end

    subgraph "supervins_loop_fusion_node (Loop Closure)"
        direction TB
        LoopEntry["Loop Fusion Node\npose_graph_node.cpp"]:::ros
        KeyframeManager["Keyframe Manager\nKeyframe.cpp/.h"]:::processing
        SuperPointExt["SuperPoint Extractor"]:::processing
        DBow3DB["DBoW3 Database"]:::external
        BriefSupport["BRIEF Support Files"]:::external
        LoopDetector["Loop Detector"]:::processing
        PoseGraphOpt["Pose-Graph Optimizer"]:::optimization
        UtilityLoop["Utility & Visualization Helpers"]:::processing
        PoseGraphLogs["Pose Graph Output Logs\npose_graph_output.txt"]:::ros
        LoopEntry --> KeyframeManager
        KeyframeManager --> SuperPointExt
        SuperPointExt --> DBow3DB
        DBow3DB --> LoopDetector
        LoopDetector --> PoseGraphOpt
        PoseGraphOpt --> PoseGraphLogs
        UtilityLoop --> PoseGraphLogs
        FeatureTracker -.-> KeyframeManager
    end

    %% Shared Modules & Resources
    CameraModels["camera_models Module"]:::external
    ConfigFiles["Configuration Files\n(config)"]:::external
    WeightsDPL["ONNX Models\nweights_dpl"]:::external
    SupportFiles["support_files"]:::external
    Resources["resources"]:::external

    CameraModels --> Initializer
    CameraModels --> CeresOptimizer
    CameraModels --> PoseGraphOpt
    ConfigFiles -.-> Entry
    WeightsDPL -.-> SPDetector
    SupportFiles --> BriefSupport
    Resources -.-> LaunchRViz

    %% External Dependencies
    CeresLib["Ceres Solver"]:::external
    EigenLib["Eigen"]:::external
    OpenCVLib["OpenCV"]:::external
    ORTLib["ONNX Runtime"]:::external
    ROSLib["ROS Noetic"]:::external
    DBoW3Lib["DBoW3"]:::external

    CeresLib --> CeresOptimizer
    EigenLib --> CeresOptimizer
    OpenCVLib --> SPDetector
    ORTLib --> SPDetector
    ORTLib --> LGMatcher
    ROSLib --> Entry
    ROSLib --> LoopEntry
    DBoW3Lib --> DBow3DB

    %% Legend
    subgraph "Legend"
        direction TB
        L1["Blue: ROS nodes & topics"]:::ros
        L2["Green: Processing modules"]:::processing
        L3["Orange: Optimization engines"]:::optimization
        L4["Gray: External libs & resources"]:::external
    end

    %% Click events
    click CameraModels "https://github.com/luohongk/supervins/tree/main/camera_models"
    click ConfigFiles "https://github.com/luohongk/supervins/tree/main/config"
    click WeightsDPL "https://github.com/luohongk/supervins/tree/main/supervins_estimator/weights_dpl"
    click Entry "https://github.com/luohongk/supervins/blob/main/supervins_estimator/src/supervins_main.cpp"
    click FeatureTracker "https://github.com/luohongk/supervins/tree/main/supervins_estimator/src/featureTracker"
    click Initializer "https://github.com/luohongk/supervins/tree/main/supervins_estimator/src/initial"
    click CeresOptimizer "https://github.com/luohongk/supervins/tree/main/supervins_estimator/src/factor"
    click UtilityEst "https://github.com/luohongk/supervins/tree/main/supervins_estimator/src/utility"
    click LaunchRViz "https://github.com/luohongk/supervins/blob/main/supervins_estimator/launch/supervins_rviz.launch"
    click VIOLogs "https://github.com/luohongk/supervins/blob/main/supervins_estimator/vio_output/vio_output.txt"
    click LoopEntry "https://github.com/luohongk/supervins/blob/main/supervins_loop_fusion/src/pose_graph_node.cpp"
    click KeyframeManager "https://github.com/luohongk/supervins/blob/main/supervins_loop_fusion/src/keyframe.cpp"
    click DBow3DB "https://github.com/luohongk/supervins/tree/main/supervins_loop_fusion/src/ThirdParty/DBoW3"
    click BriefSupport "https://github.com/luohongk/supervins/blob/main/supervins_loop_fusion/src/ThirdParty/DVision/BRIEF.cpp"
    click PoseGraphOpt "https://github.com/luohongk/supervins/blob/main/supervins_loop_fusion/src/pose_graph.cpp"
    click UtilityLoop "https://github.com/luohongk/supervins/tree/main/supervins_loop_fusion/src/utility"
    click PoseGraphLogs "https://github.com/luohongk/supervins/blob/main/supervins_loop_fusion/pose_graph_output/pose_graph_output.txt"

    %% Styles
    classDef ros fill:#B3E5FC,stroke:#333,stroke-width:1px;
    classDef processing fill:#C8E6C9,stroke:#333,stroke-width:1px;
    classDef optimization fill:#FFE0B2,stroke:#333,stroke-width:1px;
    classDef external fill:#E0E0E0,stroke:#333,stroke-width:1px;
```
