How vladyslavusenko/basalt works
Project Type and Purpose
• This project is a research-oriented visual-inertial odometry and mapping system that includes utilities for camera/IMU/mocap calibration, visual odometry (with and without inertial data), and simulation tools. It is implemented primarily in C++ (using CMake for builds) with a supplementary Python interface for experiments and evaluation.
• The README also mentions a header-only library version (basalt-headers) that contains several reusable components.
Key Architectural Components to Highlight
a. Calibration Tools:
• Modules under directories such as “include/basalt/calibration” and “src/calibration” implement camera, IMU, and motion capture calibration.
• They include files for camera model calibration (e.g., aprilgrid, vignette correction) and calibration helpers.
b. Visual-Inertial Odometry (VIO) and Visual Odometry (VO):
• The core VIO pipeline is in “include/basalt/vi_estimator” and “src/vi_estimator” for estimation and optimization.
• The VO parts (without IMU data) are supported as separate routines (e.g., src/vo.cpp and dataset I/O modules).
c. Data Input/Output and Dataset Handling:
• “include/basalt/io” and “src/io” modules provide functionality to read datasets such as Euroc, KITTI, TUMVI, and UZH.
• These modules abstract the dataset interfaces and provide sensor data (camera images, IMU measurements, etc.) to the pipelines.
d. Optimization and Linearization Modules:
• Components in “include/basalt/linearization” and “src/linearization” implement different block-permutation linearizations, such as square-root and QR-based formulations as well as marginalization, reflecting the optimization backbone.
e. Mapping and Simulation:
• The “src/mapper”, “src/mapper_sim”, and “src/mapper_sim_naive” components are responsible for map building and simulation of the estimation processes.
• Simulation modules also exist to test various components in controlled settings.
f. Device Support:
• Specific modules (e.g., “include/basalt/device/rs_t265.h” and “src/device/rs_t265.cpp”) provide support for devices such as the Intel RealSense T265.
g. External Integrations and Dependencies:
• The project uses several third-party libraries, as evidenced in the “thirdparty” folder (e.g., Pangolin for visualization, apriltag for tag detection, Eigen, OpenGV, TBB, ROS, and even CLI11 for command-line parsing).
• The build and configuration system (CMake, .clang-format, Dockerfiles) indicate an emphasis on cross-platform compatibility and reproducible builds.
h. Evaluation and Batch Processing:
• Python scripts under “python/basalt” and shell scripts under “scripts/batch” facilitate batch evaluations and experiment management.
• There are dedicated tools for generating reports/latex outputs.
Relationships and Interactions
• From sensor/dataset input (Layer 1) to calibration modules (Layer 2).
• Calibration outputs feed into both VO and VIO pipelines (Layer 3).
• Within the core processing, illustrate the iterative loop between the pose estimation modules and the optimization backends (linearization and marginalization).
• Out feedback loops common in optimization (e.g., sliding window BA and recurrent marginalization).
• Integration with third-party libraries and external submodules, emphasizing dependencies (e.g., drawing borders or labels that show “external libs: Pangolin, Eigen, etc.”).