# princeton-vl/dpvo architecture

> Architecture diagram of the GitHub repository princeton-vl/dpvo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/princeton-vl/dpvo
- Repository: https://github.com/princeton-vl/dpvo

## Overview

##### Project type and purpose
- This is a research‐oriented, full-stack SLAM/visual odometry system combining Python, C++/CUDA, deep learning, and classical vision modules. It provides:
- A core DPVO package (deep patch odometry & SLAM)
- A visualization component (DPViewer + Pangolin)
- Optional classical loop-closure (DBoW2 + DPRetrieval)
- Training, evaluation, and demo scripts

##### File structure
- Top-level directories:
- dpvo/                ← Python package for core pipeline
- DPViewer/            ← Python/C++ viewer package (pybind11 + Pangolin)
- DBoW2/ & DPRetrieval/← classical loop‐closure C++ + Python bindings
- Pangolin/            ← custom build scripts for 3D viewer
- config/, calib/, datasets/ ← static configs, calibration, ground-truth
- demo.py, train.py, evaluate_*.py ← entry-point scripts
- Within dpvo/:
- altcorr/    ← CUDA/C++ kernels for patch correlation
- extractor.py, net.py ← deep network definition (PyTorch)
- fastba/     ← CUDA/C++ bundle adjustment backend
- lietorch/   ← C++/CUDA Lie-groups bindings via pybind11
- loop_closure/ ← high-level SLAM back-end logic (deep+classical)
- data_readers/ ← dataset loaders & augmentation
- Build/config files: CMakeLists.txt, setup.py, environment.yml

##### README
- Confirms core components: Python DPVO API, PyTorch nets, C++/CUDA acceleration, visualizer (Pangolin), classical loop closure (DBoW2 + DPRetrieval).
- Describes user workflows: demo → processing → optional SLAM back-end → DPViewer; train → network training; evaluate → batch evaluation

1. Entry-point layer (Python scripts: demo.py, train.py, evaluate_*.py)
2. Core DPVO pipeline (Python modules):
- DataReaders
- DeepNet (extractor.py + net.py)
- PatchCorr (altcorr C++/CUDA)
- FastBA (fastba C++/CUDA)
- LieTorch (C++/CUDA Lie algebra ops)
- LoopClosure logic (deep + optional classical)
3. External classical backend (optional):
- DBoW2 C++ library ←→ DPRetrieval pybind11
4. Visualization backend:
- DPViewer Python API ←→ pybind11 ←→ Pangolin C++ renderer
5. Persistent artifacts:
- Calibration/Config files
- Datasets (images + ground-truth)
- Pretrained models (download_models_and_data.sh)
- Logs & outputs (runs/, logs/, saved .ply/.txt)

- Scripts → load config + dataset → call DPVO API
- DPVO API sequentially:
DataReaders → DeepNet feature extraction → PatchCorr → FastBA optimization → LoopClosure
- LoopClosure → (if CLASSIC_LOOP_CLOSURE) call DPRetrieval → DBoW2 → back to LoopClosure
- Final poses/point-cloud → (if viz flag) → DPViewer → Pangolin display
- Training scripts → DPVO API + DeepNet → optimizer → save model → log
- Evaluation scripts → DPVO API → compute metrics → log/plot

c. Architectural patterns & principles
- Modular “processing pipeline” → each stage isolated (reader, network, corr, BA, closure)
- High-performance modules in C++/CUDA exposed via pybind11
- Optional classical extension (plug-in style)
- Python “orchestration layer” for ease of scripting and experiment management

d. Key technologies/frameworks
- Python 3.8+, PyTorch, numpy, ruamel.yaml
- C++17, CUDA kernels, Eigen
- pybind11 (for all C++↔Python bindings: altcorr, fastba, lietorch, DPRetrieval, DPViewer)
- Pangolin for 3D rendering
- OpenCV (optional classical backend)
- DBoW2 for bag-of-words retrieval

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Entry Scripts
    subgraph "Entry Scripts"
        Demo("demo.py"):::python
        Train("train.py"):::python
        Eval("evaluate_*.py"):::python
    end

    %% DPVO Orchestration
    subgraph "DPVO Orchestration"
        Orchestration["dpvo.py / config.py / utils"]:::python
    end

    Demo --> Orchestration
    Train --> Orchestration
    Eval --> Orchestration

    %% DPVO Pipeline
    subgraph "DPVO Pipeline"
        Readers("DataReaders"):::python
        Extractor("DeepNet\n(extractor.py & net.py)"):::python
        PatchCorr("PatchCorr\n(altcorr)"):::cpp
        FastBA("FastBA\n(fastba)"):::cpp
        LieTorch("LieTorch\n(lietorch)"):::cpp
        LoopClosure("LoopClosure"):::python
    end

    Orchestration --> Readers --> Extractor --> PatchCorr --> FastBA --> LieTorch --> LoopClosure

    %% Classical Loop-Closure (Optional)
    subgraph "Classical Loop-Closure (Optional)"
        DPRetrieval("DPRetrieval"):::python
        DBoW2("DBoW2"):::external
    end

    LoopClosure --> DPRetrieval --> DBoW2 --> LoopClosure

    %% Visualization Backend
    subgraph "Visualization"
        DPViewer("DPViewer"):::python
        Pangolin("Pangolin"):::external
    end

    LoopClosure -->|"poses/point-cloud"| DPViewer --> Pangolin

    %% Data and Config
    subgraph "Configs & Data"
        Configs("config/ & calib/"):::data
        Datasets("datasets/"):::data
        Models("download_models_and_data.sh"):::data
        Logs("logs/ & runs/"):::data
    end

    Demo --> Configs
    Demo --> Datasets
    Train --> Models
    LoopClosure --> Logs
    Eval --> Logs

    %% Click Events
    click Demo "https://github.com/princeton-vl/dpvo/blob/main/demo.py"
    click Train "https://github.com/princeton-vl/dpvo/blob/main/train.py"
    click Eval "https://github.com/princeton-vl/dpvo/blob/main/evaluate_euroc.py"
    click Readers "https://github.com/princeton-vl/dpvo/tree/main/dpvo/data_readers/"
    click Extractor "https://github.com/princeton-vl/dpvo/blob/main/dpvo/extractor.py"
    click PatchCorr "https://github.com/princeton-vl/dpvo/tree/main/dpvo/altcorr/"
    click FastBA "https://github.com/princeton-vl/dpvo/tree/main/dpvo/fastba/"
    click LieTorch "https://github.com/princeton-vl/dpvo/tree/main/dpvo/lietorch/"
    click LoopClosure "https://github.com/princeton-vl/dpvo/tree/main/dpvo/loop_closure/"
    click DBoW2 "https://github.com/princeton-vl/dpvo/tree/main/DBoW2/"
    click DPRetrieval "https://github.com/princeton-vl/dpvo/tree/main/DPRetrieval/"
    click DPViewer "https://github.com/princeton-vl/dpvo/tree/main/DPViewer/"
    click Pangolin "https://github.com/princeton-vl/dpvo/tree/main/Pangolin/"
    click Configs "https://github.com/princeton-vl/dpvo/tree/main/config/"
    click Datasets "https://github.com/princeton-vl/dpvo/tree/main/datasets/"
    click Models "https://github.com/princeton-vl/dpvo/blob/main/download_models_and_data.sh"
    click Logs "https://github.com/princeton-vl/dpvo/tree/main/logs/"

    %% Styles
    classDef python fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef cpp fill:#FFA500,stroke:#333,stroke-width:1px
    classDef external fill:#90EE90,stroke:#333,stroke-width:1px
    classDef data fill:#D3D3D3,stroke:#333,stroke-width:1px
```
