# smartroboticslab/okvis2 architecture

> Architecture diagram of the GitHub repository smartroboticslab/okvis2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/smartroboticslab/okvis2
- Repository: https://github.com/smartroboticslab/okvis2

## Overview

##### Project type and purpose
- This is OKVIS2: an open-source, modular C++ library and set of applications for real-time keyframe-based visual-inertial SLAM (Simultaneous Localization and Mapping).
- It comprises low-level utilities, sensor-handling, frontend feature pipelines, backend optimization, and ROS2 integration, all built via CMake.

##### File structure
- Top-level folders:
- cmake / CMakeLists.txt → build configuration
- external → third-party submodules (DBoW2, opengv, Ceres, etc.)
- config / launch → YAML configs and ROS2 launch files
- okvis_apps → demo & recorder executables
- okvis_common, okvis_cv, okvis_frontend, okvis_ceres, okvis_kinematics, okvis_multisensor_processing, okvis_ros2, okvis_time, okvis_timing, okvis_util → core library modules
- resources / cnn → segmentation model artifacts and helper scripts
- Pattern: layered, modular libraries with clear dependency flows (common → cv → frontend → ceres backend; multisensor processing → frontend/backend; ros2 wrappers on top).

##### README for additional insights
- Confirms pure CMake, key dependencies (Eigen, Ceres, Glog, OpenCV, LibTorch for optional CNN), ROS2 build option.
- Describes pipeline callbacks (optimised graph & IMU callbacks) and trajectory management.
- Describes demo apps and ROS2 nodes for live processing or bag playback.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Sensor Interfaces
    subgraph "Sensor Interfaces" 
        direction TB
        DatasetReader["DatasetReader"]:::io
        RpgDatasetReader["RpgDatasetReader"]:::io
        Realsense["Realsense Driver"]:::io
        RealsenseRgbd["RealsenseRgbd Driver"]:::io
    end

    %% Optional Preprocessing
    subgraph "Optional Preprocessing" 
        direction TB
        CNNDemo["CNN Segmentation Demo"]:::io
        CNNModel["fast-scnn.pt Model"]:::io
    end

    %% Multisensor Processing
    subgraph "Multisensor Processing" 
        direction TB
        ThreadedSlam["ThreadedSlam"]:::core
        QueuedTrajectory["QueuedTrajectory"]:::core
        ThreadsafeQueue["ThreadsafeQueue"]:::core
    end

    %% VIO Frontend
    subgraph "VIO Frontend" 
        direction TB
        Frontend["Frontend"]:::core
        FBrisk["FBrisk Detector"]:::core
        StereoTriangulation["Stereo Triangulation"]:::core
        OpenGVAdapters["OpenGV Adapters"]:::core
    end

    %% VIO Backend
    subgraph "VIO Backend" 
        direction TB
        ViSlamBackend["ViSlamBackend"]:::core
        CeresErrors["Ceres Error Interfaces"]:::core
    end

    %% Trajectory Manager
    subgraph "Trajectory Manager" 
        direction TB
        ViFrontendInterface["ViFrontendInterface"]:::core
        ViInterface["ViInterface"]:::core
        TrajectoryOutputH["TrajectoryOutput (H)"]:::core
        TrajectoryOutputCPP["TrajectoryOutput (CPP)"]:::core
    end

    %% ROS2 Integration
    subgraph "ROS2 Integration" 
        direction TB
        PublisherH["Publisher"]:::int
        SubscriberH["Subscriber"]:::int
        RePublisherH["RePublisher"]:::int
        PublisherCPP["Publisher (CPP)"]:::int
        SubscriberCPP["Subscriber (CPP)"]:::int
        RePublisherCPP["RePublisher (CPP)"]:::int
        NodeRealsense["okvis_node_realsense"]:::int
        NodeSync["okvis_node_synchronous"]:::int
    end

    %% Demo Applications
    subgraph "Demo Applications" 
        direction TB
        AppSync["okvis_app_synchronous"]:::int
        AppRealsense["okvis_app_realsense"]:::int
        AppRecorder["okvis_app_realsense_recorder"]:::int
        TestDBoW2["dbow2_test"]:::int
        TestNN["nn_test"]:::int
    end

    %% Config & Launch
    subgraph "Configuration & Launch" 
        direction TB
        ConfigYAML["YAML Configs"]:::int
        RVizConfig["RViz2 Config"]:::int
        LaunchFiles["ROS2 Launch Files"]:::int
    end

    %% Visualization Assets
    subgraph "Visualization Assets" 
        direction TB
        MeshFirefly["firefly.dae"]:::int
        MeshRealsense["realsense.dae"]:::int
    end

    %% External Dependencies
    Eigen["Eigen3"]:::ext
    OpenCV["OpenCV"]:::ext
    Glog["Glog/Gflags"]:::ext
    CeresLib["Ceres & SuiteSparse"]:::ext
    DBoW2Lib["DBoW2"]:::ext
    OpenGVLib["OpenGV"]:::ext
    LibTorch["LibTorch"]:::ext
    ROS2Lib["ROS2/RViz2"]:::ext

    %% Data Flow
    DatasetReader --> ThreadedSlam
    RpgDatasetReader --> ThreadedSlam
    Realsense --> ThreadedSlam
    RealsenseRgbd --> ThreadedSlam
    ThreadedSlam --> Frontend
    QueuedTrajectory --> Frontend
    ThreadsafeQueue --> Frontend

    CNNDemo -.-> Frontend
    CNNModel -.-> CNNDemo

    Frontend --> ViSlamBackend
    StereoTriangulation --> ViSlamBackend
    OpenGVAdapters --> ViSlamBackend
    FBrisk --> Frontend

    ViSlamBackend --> ViFrontendInterface
    CeresErrors --> ViSlamBackend

    ViFrontendInterface --> TrajectoryOutputH
    ViInterface --> TrajectoryOutputH
    TrajectoryOutputH --> TrajectoryOutputCPP

    TrajectoryOutputCPP --> PublisherH
    TrajectoryOutputCPP --> SubscriberH
    TrajectoryOutputCPP --> RePublisherH

    PublisherH --> PublisherCPP
    SubscriberH --> SubscriberCPP
    RePublisherH --> RePublisherCPP

    PublisherCPP --> NodeRealsense
    SubscriberCPP --> NodeRealsense
    RePublisherCPP --> NodeRealsense
    PublisherCPP --> NodeSync
    SubscriberCPP --> NodeSync

    NodeRealsense --> AppRealsense
    NodeSync --> AppSync

    AppSync --> ConfigYAML
    AppRealsense --> ConfigYAML
    AppRecorder --> ConfigYAML

    ConfigYAML --> NodeRealsense
    ConfigYAML --> NodeSync

    NodeRealsense --> RVizConfig
    NodeSync --> RVizConfig
    RVizConfig --> ROS2Lib

    NodeRealsense --> MeshFirefly
    NodeRealsense --> MeshRealsense

    %% External dependencies connections
    Frontend -->|uses| OpenCV
    Frontend -->|uses| DBoW2Lib
    Frontend -->|uses| OpenGVLib
    ViSlamBackend -->|uses| CeresLib
    ThreadedSlam -->|links| Eigen
    CNNDemo -->|uses| LibTorch
    NodeRealsense -->|uses| ROS2Lib
    NodeSync -->|uses| ROS2Lib

    %% Click Events
    click DatasetReader "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/include/okvis/DatasetReader.hpp"
    click RpgDatasetReader "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/include/okvis/RpgDatasetReader.hpp"
    click Realsense "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/include/okvis/Realsense.hpp"
    click RealsenseRgbd "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/include/okvis/RealsenseRgbd.hpp"
    click ThreadedSlam "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/include/okvis/ThreadedSlam.hpp"
    click QueuedTrajectory "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/include/okvis/QueuedTrajectory.hpp"
    click ThreadsafeQueue "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/include/okvis/threadsafe/ThreadsafeQueue.hpp"
    click ThreadedSlam "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/src/ThreadedSlam.cpp"
    click CNNDemo "https://github.com/smartroboticslab/okvis2/blob/main/cnn/demo.py"
    click CNNModel "https://github.com/smartroboticslab/okvis2/blob/main/resources/fast-scnn.pt"
    click Frontend "https://github.com/smartroboticslab/okvis2/blob/main/okvis_frontend/include/okvis/Frontend.hpp"
    click FBrisk "https://github.com/smartroboticslab/okvis2/blob/main/okvis_frontend/include/DBoW2/FBrisk.hpp"
    click StereoTriangulation "https://github.com/smartroboticslab/okvis2/blob/main/okvis_frontend/include/okvis/triangulation/stereo_triangulation.hpp"
    click OpenGVAdapters "https://github.com/smartroboticslab/okvis2/tree/main/okvis_frontend/include/opengv"
    click ViSlamBackend "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ceres/include/okvis/ViSlamBackend.hpp"
    click CeresErrors "https://github.com/smartroboticslab/okvis2/tree/main/okvis_ceres/include/okvis/ceres"
    click ViSlamBackend "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ceres/src/ViSlamBackend.cpp"
    click ViFrontendInterface "https://github.com/smartroboticslab/okvis2/blob/main/okvis_common/include/okvis/ViFrontendInterface.hpp"
    click ViInterface "https://github.com/smartroboticslab/okvis2/blob/main/okvis_common/include/okvis/ViInterface.hpp"
    click TrajectoryOutputH "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/include/okvis/TrajectoryOutput.hpp"
    click TrajectoryOutputCPP "https://github.com/smartroboticslab/okvis2/blob/main/okvis_multisensor_processing/src/TrajectoryOutput.cpp"
    click PublisherH "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ros2/include/okvis/ros2/Publisher.hpp"
    click SubscriberH "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ros2/include/okvis/ros2/Subscriber.hpp"
    click RePublisherH "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ros2/include/okvis/ros2/RePublisher.hpp"
    click PublisherCPP "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ros2/src/Publisher.cpp"
    click SubscriberCPP "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ros2/src/Subscriber.cpp"
    click RePublisherCPP "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ros2/src/RePublisher.cpp"
    click NodeRealsense "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ros2/src/okvis_node_realsense.cpp"
    click NodeSync "https://github.com/smartroboticslab/okvis2/blob/main/okvis_ros2/src/okvis_node_synchronous.cpp"
    click AppSync "https://github.com/smartroboticslab/okvis2/blob/main/okvis_apps/src/okvis_app_synchronous.cpp"
    click AppRealsense "https://github.com/smartroboticslab/okvis2/blob/main/okvis_apps/src/okvis_app_realsense.cpp"
    click AppRecorder "https://github.com/smartroboticslab/okvis2/blob/main/okvis_apps/src/okvis_app_realsense_recorder.cpp"
    click TestDBoW2 "https://github.com/smartroboticslab/okvis2/blob/main/okvis_apps/src/dbow2_test.cpp"
    click TestNN "https://github.com/smartroboticslab/okvis2/blob/main/okvis_apps/src/nn_test.cpp"
    click ConfigYAML "https://github.com/smartroboticslab/okvis2/tree/main/config"
    click RVizConfig "https://github.com/smartroboticslab/okvis2/blob/main/config/rviz2/rviz2_config.rviz"
    click LaunchFiles "https://github.com/smartroboticslab/okvis2/tree/main/launch"
    click MeshFirefly "https://github.com/smartroboticslab/okvis2/blob/main/resources/meshes/firefly.dae"
    click MeshRealsense "https://github.com/smartroboticslab/okvis2/blob/main/resources/meshes/realsense.dae"

    %% Styles
    classDef io fill:#ccffcc,stroke:#088A29,stroke-width:1px
    classDef core fill:#cce0ff,stroke:#084B8A,stroke-width:1px
    classDef int fill:#ffe0b3,stroke:#B45F04,stroke-width:1px
    classDef ext fill:#f2f2f2,stroke:#666666,stroke-width:1px,stroke-dasharray: 2 2
```
