# rpng/open_vins architecture

> Architecture diagram of the GitHub repository rpng/open_vins, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rpng/open_vins
- Repository: https://github.com/rpng/open_vins

## Overview

1. **Repository purpose and top-level shape**
   OpenVINS is a modular visual-inertial navigation stack, organized as several C++ packages plus supporting configs, datasets, and tooling. The main runtime path is a visual-inertial estimator; surrounding packages provide feature tracking, state initialization, evaluation, and dataset support.

2. **Core estimation pipeline**
   The central data flow is: IMU measurements and camera frames enter the system, features are tracked/extracted from images, then the estimator fuses tracked features with inertial propagation to produce a continuous pose/state estimate. The estimator is implemented as a sliding-window MSCKF-style EKF, so camera features are mostly used as constraints rather than always being kept as permanent state variables.

3. **`ov_msckf`: main online estimator**
   `ov_msckf` is the runtime estimator package. Its key internal boundaries are:
   - `core/` for the high-level `VioManager` orchestration
   - `state/` for the filter state, propagation logic, and state helpers
   - `update/` for measurement update modes
   - `sim/` for simulation-driven execution
   - `ros/` for ROS1/ROS2 visualization and subscription glue  
   This package is the primary “application layer” and is where online odometry is produced.

4. **State propagation and measurement updates**
   Inside `ov_msckf`, the state is split into a propagation subsystem and several update subsystems:
   - `Propagator` handles IMU-driven prediction between camera updates
   - `UpdaterMSCKF` consumes tracked visual features
   - `UpdaterSLAM` handles persistent SLAM-style landmarks
   - `UpdaterZeroVelocity` supports stationary/vehicle-style corrections  
   This separation matters architecturally because each update mode can be enabled or combined without changing the core state container.

5. **`ov_core`: shared estimation and vision primitives**
   `ov_core` contains reusable lower-level functionality used by both initialization and filtering:
   - camera models (`CamBase`, `CamEqui`, `CamRadtan`)
   - feature tracking and management (`TrackKLT`, `TrackDescriptor`, `TrackAruco`, `FeatureDatabase`, etc.)
   - landmark and sensor data types
   - IMU/camera-related math utilities
   - simulation support such as SE(3) B-spline motion generation  
   This package is the algorithmic foundation under the estimator and should be treated as the “shared library” layer.

6. **Feature tracking boundary**
   Visual processing lives in `ov_core/src/track`. The tracker is responsible for turning raw images into feature tracks, masks, or fiducial observations before those results reach the estimator. Multiple tracker modes are supported: KLT, descriptor-based, ArUco, and simulator-driven tracking. That makes tracking a distinct subsystem from filtering, with a clean handoff at the feature-observation level.

7. **`ov_init`: bootstrapping the estimator**
   `ov_init` is the initialization layer used before or at the start of filtering. It contains:
   - static initialization
   - dynamic initialization
   - inertial initialization logic
   - simulation-aware initialization helpers
   - Ceres-based factors and local state parameterizations  
   Its role is to estimate a good initial pose/velocity/bias/extrinsic state so the MSCKF can start in a stable regime. Architecturally, it sits between raw sensor inputs and the online filter.

8. **Calibration and sensor model support**
   The repository supports calibration-related configuration for camera intrinsics, camera-to-IMU extrinsics, time offsets, and inertial intrinsics. These concerns are encoded mostly through YAML configs under `config/` and specialized factor/state code in `ov_init` and `ov_core`. This means calibration is not a separate app; it is built into the estimator’s parameter model.

9. **ROS and non-ROS deployment boundary**
   The repo is structured to build both as ROS1, ROS2, or ROS-free. `ov_msckf`, `ov_core`, `ov_init`, and `ov_eval` each have ROS-specific CMake glue and launch files where needed, but the algorithmic code is largely shared. So the architecture has a clean split between transport/integration layers and core estimator logic.

10. **`ov_eval`: offline evaluation and analysis**
   `ov_eval` is a separate analysis subsystem. It loads trajectories, aligns estimates to ground truth, computes metrics, and generates timing/error comparisons. This package does not participate in online estimation; it consumes outputs from the estimator and dataset recordings. It is the post-processing boundary of the system.

11. **`ov_data` and `config/`: dataset and experiment inputs**
   `ov_data` contains prepared trajectory/ground-truth text and CSV files for supported datasets and simulations. `config/` contains dataset-specific estimator and calibration YAMLs, plus a few masks and example launch snippets. These directories define the experiment inputs that drive which sensor models, tracking settings, and estimator options are used at runtime.

12. **Simulation support as a first-class path**
   Simulation is not an afterthought: both `ov_core` and `ov_msckf` include simulation components, and `ov_init` has simulator initialization helpers. The simulator path feeds synthetic IMU and vision data into the same estimator interfaces used by real datasets, which keeps the runtime architecture consistent across real and synthetic runs.

13. **Operational tooling and packaging**
   The repository includes Dockerfiles for multiple ROS/distros, GitHub Actions for CI, and docs/Doxygen sources. These do not change the estimator architecture, but they indicate that the system is meant to be built and exercised in several environment variants with the same underlying codebase.

14. **What to draw in an architecture diagram**
   A good diagram should show:
   - sensor inputs: camera images, IMU, optional fiducials/masks
   - feature tracking and camera modeling in `ov_core`
   - initialization in `ov_init`
   - online filtering in `ov_msckf` with propagation plus multiple update paths
   - outputs: odometry/state estimates and visualization/ROS topics
   - offline sidecars: `ov_eval` for metrics and `ov_data`/`config` for datasets and experiment settings  
   The main architectural boundary is between reusable algorithm libraries (`ov_core`, `ov_init`) and the online application runtime (`ov_msckf`), with `ov_eval` as a separate downstream consumer.

Main entry points: [`ov_msckf`](https://github.com/rpng/open_vins/tree/master/ov_msckf) (ov_msckf), [`ov_init`](https://github.com/rpng/open_vins/tree/master/ov_init) (ov_init), [`ov_core`](https://github.com/rpng/open_vins/tree/master/ov_core) (ov_core), [`ov_eval`](https://github.com/rpng/open_vins/tree/master/ov_eval) (ov_eval)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_grp_shared["Shared libraries"]
  node_node_ov_core["ov_core<br/>shared lib"]
  node_node_cam_models["Camera models"]
  node_node_tracking["Tracking"]
  node_node_features["Feature DB<br/>feature state"]
  node_node_types_utils["Types & utils<br/>math/types"]
  node_node_sim_core["Core sim<br/>simulation"]
end

subgraph group_grp_runtime["Online runtime"]
  node_node_ov_init["ov_init<br/>initialization lib"]
  node_node_init_static["Static init<br/>init stage"]
  node_node_init_dynamic["Dynamic init<br/>init stage"]
  node_node_init_inertial["Inertial init<br/>init stage"]
  node_node_ov_msckf["ov_msckf<br/>online estimator"]
  node_node_vio_manager["VIO manager<br/>orchestrator"]
  node_node_propagator["Propagator<br/>imu prediction"]
  node_node_updates["Updates"]
  node_node_ros_glue["ROS glue<br/>integration"]
  node_node_sim_runtime["Runtime sim<br/>sim runner"]
end

subgraph group_grp_offline["Offline analysis"]
  node_node_ov_eval["ov_eval<br/>analysis tool"]
  node_node_alignment["Alignment"]
  node_node_metrics["Metrics<br/>result calc"]
  node_node_loader["Loader<br/>io helper"]
end

subgraph group_grp_inputs["Experiment inputs"]
  node_node_config["config"]
  node_node_data["ov_data<br/>dataset pack"]
end

subgraph group_grp_tooling["Tooling"]
  node_node_docs["docs<br/>documentation"]
end

node_node_tracking -->|"feature tracks"| node_node_vio_manager
node_node_cam_models -->|"image geometry"| node_node_tracking
node_node_features -->|"observations"| node_node_vio_manager
node_node_types_utils -->|"sensor types"| node_node_vio_manager
node_node_ov_core -->|"provides"| node_node_tracking
node_node_ov_core -->|"provides"| node_node_features
node_node_ov_core -->|"provides"| node_node_types_utils
node_node_ov_core -->|"provides"| node_node_sim_core
node_node_init_static -->|"bootstraps"| node_node_ov_init
node_node_init_dynamic -->|"bootstraps"| node_node_ov_init
node_node_init_inertial -->|"bootstraps"| node_node_ov_init
node_node_ov_init -->|"initial state"| node_node_ov_msckf
node_node_vio_manager -->|"coordinates"| node_node_propagator
node_node_vio_manager -->|"coordinates"| node_node_updates
node_node_propagator -->|"predict then correct"| node_node_updates
node_node_ros_glue -->|"runtime I/O"| node_node_ov_msckf
node_node_sim_runtime -->|"synthetic sensors"| node_node_ov_msckf
node_node_config -->|"settings"| node_node_ov_msckf
node_node_config -->|"calibration"| node_node_ov_init
node_node_data -->|"datasets"| node_node_ov_msckf
node_node_ov_msckf -->|"outputs"| node_node_ov_eval
node_node_ov_eval -->|"aligns"| node_node_alignment
node_node_ov_eval -->|"computes"| node_node_metrics
node_node_ov_eval -->|"loads"| node_node_loader
node_node_docs -->|"documents"| node_node_ov_core
node_node_docs -->|"documents"| node_node_ov_init
node_node_docs -->|"documents"| node_node_ov_msckf
node_node_docs -->|"documents"| node_node_ov_eval

click node_node_ov_core "https://github.com/rpng/open_vins/tree/master/ov_core"
click node_node_cam_models "https://github.com/rpng/open_vins/tree/master/ov_core/src/cam"
click node_node_tracking "https://github.com/rpng/open_vins/tree/master/ov_core/src/track"
click node_node_features "https://github.com/rpng/open_vins/tree/master/ov_core/src/feat"
click node_node_types_utils "https://github.com/rpng/open_vins/tree/master/ov_core/src/types"
click node_node_sim_core "https://github.com/rpng/open_vins/tree/master/ov_core/src/sim"
click node_node_ov_init "https://github.com/rpng/open_vins/tree/master/ov_init"
click node_node_init_static "https://github.com/rpng/open_vins/tree/master/ov_init/src/static"
click node_node_init_dynamic "https://github.com/rpng/open_vins/tree/master/ov_init/src/dynamic"
click node_node_init_inertial "https://github.com/rpng/open_vins/tree/master/ov_init/src/init"
click node_node_ov_msckf "https://github.com/rpng/open_vins/tree/master/ov_msckf"
click node_node_vio_manager "https://github.com/rpng/open_vins/tree/master/ov_msckf/src/core"
click node_node_propagator "https://github.com/rpng/open_vins/tree/master/ov_msckf/src/state"
click node_node_updates "https://github.com/rpng/open_vins/tree/master/ov_msckf/src/update"
click node_node_ros_glue "https://github.com/rpng/open_vins/tree/master/ov_msckf/src/ros"
click node_node_sim_runtime "https://github.com/rpng/open_vins/tree/master/ov_msckf/src/sim"
click node_node_ov_eval "https://github.com/rpng/open_vins/tree/master/ov_eval"
click node_node_alignment "https://github.com/rpng/open_vins/tree/master/ov_eval/src/alignment"
click node_node_metrics "https://github.com/rpng/open_vins/tree/master/ov_eval/src/calc"
click node_node_loader "https://github.com/rpng/open_vins/tree/master/ov_eval/src/utils"
click node_node_config "https://github.com/rpng/open_vins/tree/master/config"
click node_node_data "https://github.com/rpng/open_vins/tree/master/ov_data"
click node_node_docs "https://github.com/rpng/open_vins/tree/master/docs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_ov_core,node_node_cam_models,node_node_tracking,node_node_features,node_node_types_utils,node_node_sim_core toneBlue
class node_node_ov_init,node_node_init_static,node_node_init_dynamic,node_node_init_inertial,node_node_ov_msckf,node_node_vio_manager,node_node_propagator,node_node_updates,node_node_ros_glue,node_node_sim_runtime toneAmber
class node_node_ov_eval,node_node_alignment,node_node_metrics,node_node_loader toneMint
class node_node_config,node_node_data toneRose
class node_node_docs toneIndigo
```

## Components

### Shared libraries

Reusable estimation and vision primitives

- [ov_core](https://github.com/rpng/open_vins/tree/master/ov_core): shared lib · Camera models, tracking, features, sensor types, and math utilities · `ov_core`
- [Camera models](https://github.com/rpng/open_vins/tree/master/ov_core/src/cam): camera model · Pinhole and fisheye camera abstractions · `ov_core/src/cam`
- [Tracking](https://github.com/rpng/open_vins/tree/master/ov_core/src/track): vision tracking · KLT, descriptor, ArUco, and simulated feature tracking · `ov_core/src/track`
- [Feature DB](https://github.com/rpng/open_vins/tree/master/ov_core/src/feat): feature state · Feature, landmark, and track bookkeeping · `ov_core/src/feat`
- [Types & utils](https://github.com/rpng/open_vins/tree/master/ov_core/src/types): math/types · IMU, pose, landmark, and helper math/data utilities · `ov_core/src/types`
- [Core sim](https://github.com/rpng/open_vins/tree/master/ov_core/src/sim): simulation · SE(3) spline motion generation and simulator helpers · `ov_core/src/sim`

### Online runtime

Main visual-inertial estimator path

- [ov_init](https://github.com/rpng/open_vins/tree/master/ov_init): initialization lib · Static, dynamic, and inertial bootstrapping for the filter · `ov_init`
- [Static init](https://github.com/rpng/open_vins/tree/master/ov_init/src/static): init stage · Initialization for stationary starts · `ov_init/src/static`
- [Dynamic init](https://github.com/rpng/open_vins/tree/master/ov_init/src/dynamic): init stage · Initialization for moving-platform starts · `ov_init/src/dynamic`
- [Inertial init](https://github.com/rpng/open_vins/tree/master/ov_init/src/init): init stage · Bias, gravity, and motion initialization logic · `ov_init/src/init`
- [ov_msckf](https://github.com/rpng/open_vins/tree/master/ov_msckf): online estimator · Sliding-window visual-inertial odometry runtime · `ov_msckf`
- [VIO manager](https://github.com/rpng/open_vins/tree/master/ov_msckf/src/core): orchestrator · High-level runtime coordination and estimator lifecycle · `ov_msckf/src/core`
- [Propagator](https://github.com/rpng/open_vins/tree/master/ov_msckf/src/state): imu prediction · IMU-driven state propagation between vision updates · `ov_msckf/src/state`
- [Updates](https://github.com/rpng/open_vins/tree/master/ov_msckf/src/update): measurement updates · Visual, SLAM, and zero-velocity correction paths · `ov_msckf/src/update`
- [ROS glue](https://github.com/rpng/open_vins/tree/master/ov_msckf/src/ros): integration · ROS1/ROS2 subscriptions, visualization, and launch entrypoints · `ov_msckf/src/ros`
- [Runtime sim](https://github.com/rpng/open_vins/tree/master/ov_msckf/src/sim): sim runner · Synthetic sensor generation and estimator exercise · `ov_msckf/src/sim`

### Offline analysis

Evaluation and post-processing

- [ov_eval](https://github.com/rpng/open_vins/tree/master/ov_eval): analysis tool · Trajectory alignment, metrics, timing, and plotting · `ov_eval`
- [Alignment](https://github.com/rpng/open_vins/tree/master/ov_eval/src/alignment): traj alignment · Estimate/ground-truth alignment and comparison utilities · `ov_eval/src/alignment`
- [Metrics](https://github.com/rpng/open_vins/tree/master/ov_eval/src/calc): result calc · Error, timing, and trajectory result computation · `ov_eval/src/calc`
- [Loader](https://github.com/rpng/open_vins/tree/master/ov_eval/src/utils): io helper · Input loading and recording helpers for evaluation · `ov_eval/src/utils`

### Experiment inputs

Dataset and configuration artifacts

- [config](https://github.com/rpng/open_vins/tree/master/config): experiment config · Estimator, calibration, masks, and launch examples · `config`
- [ov_data](https://github.com/rpng/open_vins/tree/master/ov_data): dataset pack · Prepared trajectories and ground-truth files for datasets and simulation · `ov_data`

### Tooling

Build, docs, and environment support

- [docs](https://github.com/rpng/open_vins/tree/master/docs): documentation · Developer, calibration, evaluation, and simulation documentation · `docs`

## Connections

- Tracking → VIO manager: feature tracks · Tracked image observations are handed to the estimator
- Camera models → Tracking: image geometry · Tracking uses camera projection and distortion models
- Feature DB → VIO manager: observations · Feature records flow into the estimator update logic
- Types & utils → VIO manager: sensor types · Shared state and sensor representations support runtime estimation
- ov_core → Tracking: provides · Core library supplies the tracking subsystem
- ov_core → Feature DB: provides · Core library supplies feature and landmark primitives
- ov_core → Types & utils: provides · Core library supplies sensor and math utilities
- ov_core → Core sim: provides · Core library supplies shared simulation support
- Static init → ov_init: bootstraps · Static initialization is part of the initialization library
- Dynamic init → ov_init: bootstraps · Dynamic initialization is part of the initialization library
- Inertial init → ov_init: bootstraps · Inertial initialization is part of the initialization library
- ov_init → ov_msckf: initial state · Initialization provides the starting state for filtering
- VIO manager → Propagator: coordinates · Manager invokes propagation inside the estimator loop
- VIO manager → Updates: coordinates · Manager selects and sequences measurement updates
- Propagator → Updates: predict then correct · IMU prediction feeds subsequent measurement correction
- ROS glue → ov_msckf: runtime I/O · ROS integration feeds and visualizes the online estimator
- Runtime sim → ov_msckf: synthetic sensors · Simulation drives the same estimator interfaces as real data
- config → ov_msckf: settings · Estimator and calibration YAMLs configure runtime behavior
- config → ov_init: calibration · Initialization uses calibration and sensor setup inputs
- ov_data → ov_msckf: datasets · Prepared dataset trajectories drive offline or replayed runs
- ov_msckf → ov_eval: outputs · Estimated trajectories and timing results are analyzed offline
- ov_eval → Alignment: aligns · Evaluation aligns estimates to reference trajectories
- ov_eval → Metrics: computes · Evaluation computes error and timing summaries
- ov_eval → Loader: loads · Evaluation loads trajectories and recorded outputs
- docs → ov_core: documents · Docs cover the shared algorithm layer
- docs → ov_init: documents · Docs cover initialization and calibration
- docs → ov_msckf: documents · Docs cover propagation, updates, and runtime execution
- docs → ov_eval: documents · Docs cover offline metrics and evaluation workflows
