How rpng/open_vins works
1. Repository purpose and top-level shape
OpenVINS is a modular visual-inertial navigation stack, organized as several C++ packages plus supporting configs, datasets, and tooling. The main runtime path is a visual-inertial estimator; surrounding packages provide feature tracking, state initialization, evaluation, and dataset support.
2. Core estimation pipeline
The central data flow is: IMU measurements and camera frames enter the system, features are tracked/extracted from images, then the estimator fuses tracked features with inertial propagation to produce a continuous pose/state estimate. The estimator is implemented as a sliding-window MSCKF-style EKF, so camera features are mostly used as constraints rather than always being kept as permanent state variables.
3. ov_msckf: main online estimator
ov_msckf is the runtime estimator package. Its key internal boundaries are:
- core/ for the high-level VioManager orchestration
- state/ for the filter state, propagation logic, and state helpers
- update/ for measurement update modes
- sim/ for simulation-driven execution
- ros/ for ROS1/ROS2 visualization and subscription glue
This package is the primary “application layer” and is where online odometry is produced.
4. State propagation and measurement updates
Inside ov_msckf, the state is split into a propagation subsystem and several update subsystems:
- Propagator handles IMU-driven prediction between camera updates
- UpdaterMSCKF consumes tracked visual features
- UpdaterSLAM handles persistent SLAM-style landmarks
- UpdaterZeroVelocity supports stationary/vehicle-style corrections
This separation matters architecturally because each update mode can be enabled or combined without changing the core state container.
5. ov_core: shared estimation and vision primitives
ov_core contains reusable lower-level functionality used by both initialization and filtering:
- camera models (CamBase, CamEqui, CamRadtan)
- feature tracking and management (TrackKLT, TrackDescriptor, TrackAruco, FeatureDatabase, etc.)
- landmark and sensor data types
- IMU/camera-related math utilities
- simulation support such as SE(3) B-spline motion generation
This package is the algorithmic foundation under the estimator and should be treated as the “shared library” layer.
6. Feature tracking boundary
Visual processing lives in ov_core/src/track. The tracker is responsible for turning raw images into feature tracks, masks, or fiducial observations before those results reach the estimator. Multiple tracker modes are supported: KLT, descriptor-based, ArUco, and simulator-driven tracking. That makes tracking a distinct subsystem from filtering, with a clean handoff at the feature-observation level.
7. ov_init: bootstrapping the estimator
ov_init is the initialization layer used before or at the start of filtering. It contains:
- static initialization
- dynamic initialization
- inertial initialization logic
- simulation-aware initialization helpers
- Ceres-based factors and local state parameterizations
Its role is to estimate a good initial pose/velocity/bias/extrinsic state so the MSCKF can start in a stable regime. Architecturally, it sits between raw sensor inputs and the online filter.
8. Calibration and sensor model support
The repository supports calibration-related configuration for camera intrinsics, camera-to-IMU extrinsics, time offsets, and inertial intrinsics. These concerns are encoded mostly through YAML configs under config/ and specialized factor/state code in ov_init and ov_core. This means calibration is not a separate app; it is built into the estimator’s parameter model.
9. ROS and non-ROS deployment boundary
The repo is structured to build both as ROS1, ROS2, or ROS-free. ov_msckf, ov_core, ov_init, and ov_eval each have ROS-specific CMake glue and launch files where needed, but the algorithmic code is largely shared. So the architecture has a clean split between transport/integration layers and core estimator logic.
10. ov_eval: offline evaluation and analysis
ov_eval is a separate analysis subsystem. It loads trajectories, aligns estimates to ground truth, computes metrics, and generates timing/error comparisons. This package does not participate in online estimation; it consumes outputs from the estimator and dataset recordings. It is the post-processing boundary of the system.
11. ov_data and config/: dataset and experiment inputs
ov_data contains prepared trajectory/ground-truth text and CSV files for supported datasets and simulations. config/ contains dataset-specific estimator and calibration YAMLs, plus a few masks and example launch snippets. These directories define the experiment inputs that drive which sensor models, tracking settings, and estimator options are used at runtime.
12. Simulation support as a first-class path
Simulation is not an afterthought: both ov_core and ov_msckf include simulation components, and ov_init has simulator initialization helpers. The simulator path feeds synthetic IMU and vision data into the same estimator interfaces used by real datasets, which keeps the runtime architecture consistent across real and synthetic runs.
13. Operational tooling and packaging
The repository includes Dockerfiles for multiple ROS/distros, GitHub Actions for CI, and docs/Doxygen sources. These do not change the estimator architecture, but they indicate that the system is meant to be built and exercised in several environment variants with the same underlying codebase.
14. What to draw in an architecture diagram
A good diagram should show:
- sensor inputs: camera images, IMU, optional fiducials/masks
- feature tracking and camera modeling in ov_core
- initialization in ov_init
- online filtering in ov_msckf with propagation plus multiple update paths
- outputs: odometry/state estimates and visualization/ROS topics
- offline sidecars: ov_eval for metrics and ov_data/config for datasets and experiment settings
The main architectural boundary is between reusable algorithm libraries (ov_core, ov_init) and the online application runtime (ov_msckf), with ov_eval as a separate downstream consumer.
Main entry points: ov_msckf ov_msckf · ov_init ov_init · ov_core ov_core · ov_eval ov_eval
Components 23
Shared libraries
Reusable estimation and vision primitives
- ov_core · Camera models, tracking, features, sensor types, and math utilities
ov_core - Camera models · Pinhole and fisheye camera abstractions
ov_core/src/cam - Tracking · KLT, descriptor, ArUco, and simulated feature tracking
ov_core/src/track - Feature DB · Feature, landmark, and track bookkeeping
ov_core/src/feat - Types & utils · IMU, pose, landmark, and helper math/data utilities
ov_core/src/types - Core sim · SE(3) spline motion generation and simulator helpers
ov_core/src/sim
Online runtime
Main visual-inertial estimator path
- ov_init · Static, dynamic, and inertial bootstrapping for the filter
ov_init - Static init · Initialization for stationary starts
ov_init/src/static - Dynamic init · Initialization for moving-platform starts
ov_init/src/dynamic - Inertial init · Bias, gravity, and motion initialization logic
ov_init/src/init - ov_msckf · Sliding-window visual-inertial odometry runtime
ov_msckf - VIO manager · High-level runtime coordination and estimator lifecycle
ov_msckf/src/core - Propagator · IMU-driven state propagation between vision updates
ov_msckf/src/state - Updates · Visual, SLAM, and zero-velocity correction paths
ov_msckf/src/update - ROS glue · ROS1/ROS2 subscriptions, visualization, and launch entrypoints
ov_msckf/src/ros - Runtime sim · Synthetic sensor generation and estimator exercise
ov_msckf/src/sim
Offline analysis
Evaluation and post-processing
- ov_eval · Trajectory alignment, metrics, timing, and plotting
ov_eval - Alignment · Estimate/ground-truth alignment and comparison utilities
ov_eval/src/alignment - Metrics · Error, timing, and trajectory result computation
ov_eval/src/calc - Loader · Input loading and recording helpers for evaluation
ov_eval/src/utils
Experiment inputs
Dataset and configuration artifacts
Tooling
Build, docs, and environment support
- docs · Developer, calibration, evaluation, and simulation documentation
docs
Connections 28
- Tracking → VIO manager: feature tracks · Tracked image observations are handed to the estimator
- Camera models → Tracking: image geometry · Tracking uses camera projection and distortion models
- Feature DB → VIO manager: observations · Feature records flow into the estimator update logic
- Types & utils → VIO manager: sensor types · Shared state and sensor representations support runtime estimation
- ov_core → Tracking: provides · Core library supplies the tracking subsystem
- ov_core → Feature DB: provides · Core library supplies feature and landmark primitives
- ov_core → Types & utils: provides · Core library supplies sensor and math utilities
- ov_core → Core sim: provides · Core library supplies shared simulation support
- Static init → ov_init: bootstraps · Static initialization is part of the initialization library
- Dynamic init → ov_init: bootstraps · Dynamic initialization is part of the initialization library
- Inertial init → ov_init: bootstraps · Inertial initialization is part of the initialization library
- ov_init → ov_msckf: initial state · Initialization provides the starting state for filtering
- VIO manager → Propagator: coordinates · Manager invokes propagation inside the estimator loop
- VIO manager → Updates: coordinates · Manager selects and sequences measurement updates
- Propagator → Updates: predict then correct · IMU prediction feeds subsequent measurement correction
- ROS glue → ov_msckf: runtime I/O · ROS integration feeds and visualizes the online estimator
- Runtime sim → ov_msckf: synthetic sensors · Simulation drives the same estimator interfaces as real data
- config → ov_msckf: settings · Estimator and calibration YAMLs configure runtime behavior
- config → ov_init: calibration · Initialization uses calibration and sensor setup inputs
- ov_data → ov_msckf: datasets · Prepared dataset trajectories drive offline or replayed runs
- ov_msckf → ov_eval: outputs · Estimated trajectories and timing results are analyzed offline
- ov_eval → Alignment: aligns · Evaluation aligns estimates to reference trajectories
- ov_eval → Metrics: computes · Evaluation computes error and timing summaries
- ov_eval → Loader: loads · Evaluation loads trajectories and recorded outputs
- docs → ov_core: documents · Docs cover the shared algorithm layer
- docs → ov_init: documents · Docs cover initialization and calibration
- docs → ov_msckf: documents · Docs cover propagation, updates, and runtime execution
- docs → ov_eval: documents · Docs cover offline metrics and evaluation workflows