# swatishirke/visual-inertial-odometry architecture

> Architecture diagram of the GitHub repository swatishirke/visual-inertial-odometry, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/swatishirke/visual-inertial-odometry
- Repository: https://github.com/swatishirke/visual-inertial-odometry

## Overview

##### Project Type and Purpose
- This is a research‐oriented codebase for Visual-Inertial Odometry (VIO). It implements two parallel pipelines:
a. A classical, tightly-coupled MSCKF (Multi-State Constraint Kalman Filter) VIO (in Code/VIO_Kalman).
b. Deep learning based pose estimation (in Code/Deep_VIO), with separate CNN+LSTM and FlowNet sub-modules for IO (inertial odometry), VO (visual odometry) and full VIO.

##### File Structure Analysis
Top-level: “Code/” contains two directories plus a root README.
A. Code/VIO_Kalman
- vio.py (entry point)
- msckf.py (EKF/VIO core)
- config.py (parameter definitions)
- dataset.py (EuRoC data loader)
- feature.py, image.py (feature extraction & image handling)
- utils.py, viewer.py (support & visualization)
- test/ (unit tests for feature initialization & utils)
- imgs/ (example output plots)
B. Code/Deep_VIO
- CNN_LSTM/
– Misc/DataUtils.py, Misc/MiscUtils.py (data prep & helper functions)
– Network.py (model definition)
– Train_IO.py, Train_VO.py, Train_VIO.py (training pipelines)
– Test_IO.py, Test_VO.py, Test_VIO.py (inference pipelines)
- Flownet/ (same structure as CNN_LSTM under a different architecture)
- models/ (pretrained .ckpt files under subfolders IO, VO, VIO)
- images/ (example trajectories & result plots)
- report.pdf (detailed network architectures & results)

##### Key README Insights
- Two main algorithms: EKF-based MSCKF (Python reimplementation of stereo S-MSCKF) and Deep Neural Network based approaches.
- Dataset: EuRoC MAV (stereo images + IMU).
- Dependencies: numpy, scipy, opencv, pangolin (optional viewer).
- Run instructions for each pipeline.
- Results showcased via plots in imgs/.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion Layer
    subgraph "Data Ingestion Layer"
        DL["EuRoC Dataset Loader"]:::datastore
        SD["Synthetic Data Generator"]:::datastore
    end

    %% Pre-processing & Feature Extraction
    subgraph "Pre-processing & Feature Extraction"
        Sync["Image & IMU Sync & Normalization"]:::process
        FT["Feature Detector / Tracker"]:::process
        IH["Image Handling & Normalization"]:::process
    end

    %% Classical VIO Pipeline
    subgraph "Classical VIO (VIO_Kalman)"
        VIO_ENTRY["vio.py (Entry Point)"]:::service
        MSCKF["msckf.py (State Estimator & EKF Core)"]:::service
        CFG["config.py (Parameters)"]:::service
        UTIL["utils.py (Utilities)"]:::service
        VWR["viewer.py (3D Trajectory Viewer)"]:::service
        IMGS["imgs/ (Output Plots)"]:::datastore
        TESTF["test/feature_initialization_test.py"]:::script
        TESTU["test/utils_test.py"]:::script
    end

    %% Deep VIO Pipeline
    subgraph "Deep VIO"
        %% CNN+LSTM Branch
        subgraph "CNN_LSTM Branch"
            DL_Data["DataUtils.py"]:::service
            MU["MiscUtils.py"]:::service
            NNET["Network.py"]:::service
            TR_IO["Train_IO.py"]:::script
            TR_VO["Train_VO.py"]:::script
            TR_VIO["Train_VIO.py"]:::script
            TE_IO["Test_IO.py"]:::script
            TE_VO["Test_VO.py"]:::script
            TE_VIO["Test_VIO.py"]:::script
        end
        %% FlowNet Branch
        subgraph "FlowNet Branch"
            FN_Data["DataUtils.py"]:::service
            FN_MU["MiscUtils.py"]:::service
            FN_NET["Network.py"]:::service
            FN_TR_IO["Train_IO.py"]:::script
            FN_TR_VO["Train_VO.py"]:::script
            FN_TR_VIO["Train_VIO.py"]:::script
            FN_TE_IO["Test_IO.py"]:::script
            FN_TE_VO["Test_VO.py"]:::script
            FN_TE_VIO["Test_VIO.py"]:::script
            FN_HAR["test.py"]:::script
        end
    end

    %% Model Storage
    MS["models/ (Checkpoints)"]:::datastore

    %% Visualization & Reporting
    subgraph "Visualization & Reporting"
        PLOTS["images/ (Trajectories & Plots)"]:::datastore
        RPT["report.pdf"]:::report
    end

    %% Data Flow Connections
    DL --> Sync
    SD --> DL_Data
    Sync --> FT
    Sync --> IH
    FT --> MSCKF
    IH --> MSCKF

    %% Classical Pipeline Flow
    MSCKF --> VIO_ENTRY
    VIO_ENTRY --> VWR
    VWR --> IMGS
    VIO_ENTRY --> TESTF
    VIO_ENTRY --> TESTU
    VIO_ENTRY --> CFG
    VIO_ENTRY --> UTIL

    %% Deep Pipeline Flow
    DL_Data --> NNET
    MU --> NNET
    NNET --> TR_IO
    NNET --> TR_VO
    NNET --> TR_VIO
    TR_IO --> TE_IO
    TR_VO --> TE_VO
    TR_VIO --> TE_VIO
    TE_IO --> MS
    TE_VO --> MS
    TE_VIO --> MS

    FN_Data --> FN_NET
    FN_MU --> FN_NET
    FN_NET --> FN_TR_IO
    FN_NET --> FN_TR_VO
    FN_NET --> FN_TR_VIO
    FN_TR_IO --> FN_TE_IO
    FN_TR_VO --> FN_TE_VO
    FN_TR_VIO --> FN_TE_VIO
    FN_TE_IO --> MS
    FN_TE_VO --> MS
    FN_TE_VIO --> MS
    FN_HAR --> MS

    MS --> PLOTS
    MS --> RPT

    %% Click Events
    click DL "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/dataset.py"
    click FT "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/feature.py"
    click IH "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/image.py"
    click MSCKF "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/msckf.py"
    click VIO_ENTRY "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/vio.py"
    click CFG "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/config.py"
    click UTIL "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/utils.py"
    click VWR "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/viewer.py"
    click IMGS "https://github.com/swatishirke/visual-inertial-odometry/tree/main/Code/VIO_Kalman/imgs/"
    click TESTF "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/test/feature_initialization_test.py"
    click TESTU "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/VIO_Kalman/test/utils_test.py"
    click DL_Data "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/CNN_LSTM/Misc/DataUtils.py"
    click MU "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/CNN_LSTM/Misc/MiscUtils.py"
    click NNET "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/CNN_LSTM/Network.py"
    click TR_IO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/CNN_LSTM/Train_IO.py"
    click TR_VO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/CNN_LSTM/Train_VO.py"
    click TR_VIO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/CNN_LSTM/Train_VIO.py"
    click TE_IO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/CNN_LSTM/Test_IO.py"
    click TE_VO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/CNN_LSTM/Test_VO.py"
    click TE_VIO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/CNN_LSTM/Test_VIO.py"
    click FN_Data "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/Misc/DataUtils.py"
    click FN_MU "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/Misc/MiscUtils.py"
    click FN_NET "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/Network.py"
    click FN_TR_IO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/Train_IO.py"
    click FN_TR_VO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/Train_VO.py"
    click FN_TR_VIO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/Train_VIO.py"
    click FN_TE_IO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/Test_IO.py"
    click FN_TE_VO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/Test_VO.py"
    click FN_TE_VIO "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/Test_VIO.py"
    click FN_HAR "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/Flownet/test.py"
    click MS "https://github.com/swatishirke/visual-inertial-odometry/tree/main/Code/Deep_VIO/models/"
    click PLOTS "https://github.com/swatishirke/visual-inertial-odometry/tree/main/Code/Deep_VIO/images/"
    click RPT "https://github.com/swatishirke/visual-inertial-odometry/blob/main/Code/Deep_VIO/report.pdf"

    %% Styles
    classDef datastore fill:#BFD8B8,stroke:#333,stroke-width:1px;
    classDef process fill:#F9E79F,stroke:#333,stroke-width:1px;
    classDef service fill:#AED6F1,stroke:#333,stroke-width:1px;
    classDef script fill:#F5B7B1,stroke:#333,stroke-width:1px;
    classDef report fill:#D2B4DE,stroke:#333,stroke-width:1px;
```
