How uoip/stereo_msckf works
Project type and purpose
• This is a research-grade, open-source tool: a Python re-implementation of the stereo version of the Multi-State Constraint Kalman Filter (S-MSCKF) for visual-inertial odometry. Its goal is to load stereo images and IMU data (from EuRoC MAV), tightly couple them in an EKF, and output a pose trajectory (optionally visualized).
File structure
• Top–level modules:
• config.py – global parameters (camera intrinsics, noise models, EKF settings)
• dataset.py – loader/publisher for the EuRoC MAV dataset (stereo images + IMU)
• image.py – image preprocessing (undistortion, rectification)
• feature.py – feature detection/tracking and stereo matching
• msckf.py – core EKF state, propagation, update, residual building
• vio.py – main script / CLI entrypoint (argument parsing, orchestrator)
• viewer.py – optional trajectory visualization (pangolin)
• utils.py – linear algebra helpers, quaternion math, timing, config parsing
• imgs/ – sample output plots
• test/ – unit tests for feature initialization and utility functions
• No separate “frontend/backend”—this is a single executable pipeline.
README for additional insights
• Confirms stereo-MSCKF algorithm, dependencies (numpy, scipy, OpenCV, optional pangolin), dataset type, and run commands.
Main components & their relationships
a. Configuration Module (config.py)
• Defines camera calibration, IMU noise, EKF parameters
• Consumed by dataset loader, feature extractor, and MSCKF core
b. Dataset Loader & Publisher (dataset.py)
• Reads stereo image streams + time-synced IMU data
• Publishes timestamped IMU measurements & image pairs downstream
c. Image Preprocessing (image.py)
• Undistorts / rectifies raw images based on config intrinsics
• Forwards rectified images to feature module
d. Feature Processing (feature.py)
• Detects keypoints, tracks them across frames, and performs stereo match
• Builds feature tracks & measurement lists for filter update
e. MSCKF Core Filter (msckf.py)
• Maintains EKF state: IMU pose, velocity, biases + cloned camera poses
• Propagation: integrates IMU measurements to predict state
• Update: constructs measurement Jacobians from feature tracks, applies EKF update
• Manages state augmentation / marginalization
f. Visualization Module (viewer.py)
• Subscribes to updated pose stream
• Renders trajectory & keyframe poses (using pangolin if available)
g. Orchestrator / CLI (vio.py)
• Parses command-line args (path, view flag)
• Instantiates Dataset, MSCKF, Viewer
• Drives the main loop: pull IMU until next image, propagate filter, receive image, detect features, update filter, send pose to viewer
h. Utilities (utils.py)
• Shared math routines: quaternion‐vector transforms, Jacobian computations, linear solvers
Architectural patterns & design principles
• Modular pipeline: clear separation of data ingestion, preprocessing, feature extraction, estimation core, visualization.
• Publisher/subscriber (dataset publishes, MSCKF & viewer subscribe) – can be drawn as event streams.
• Stateful filter with propagation/update loop.
• Config-driven behavior: all parameters centrally defined.
• Test harness for correctness on small units (feature init, utils).