How princeton-vl/dpvo works
Project type and purpose
• This is a research‐oriented, full-stack SLAM/visual odometry system combining Python, C++/CUDA, deep learning, and classical vision modules. It provides:
• A core DPVO package (deep patch odometry & SLAM)
• A visualization component (DPViewer + Pangolin)
• Optional classical loop-closure (DBoW2 + DPRetrieval)
• Training, evaluation, and demo scripts
File structure
• Top-level directories:
• dpvo/ ← Python package for core pipeline
• DPViewer/ ← Python/C++ viewer package (pybind11 + Pangolin)
• DBoW2/ & DPRetrieval/← classical loop‐closure C++ + Python bindings
• Pangolin/ ← custom build scripts for 3D viewer
• config/, calib/, datasets/ ← static configs, calibration, ground-truth
• demo.py, train.py, evaluate_*.py ← entry-point scripts
• Within dpvo/:
• altcorr/ ← CUDA/C++ kernels for patch correlation
• extractor.py, net.py ← deep network definition (PyTorch)
• fastba/ ← CUDA/C++ bundle adjustment backend
• lietorch/ ← C++/CUDA Lie-groups bindings via pybind11
• loop_closure/ ← high-level SLAM back-end logic (deep+classical)
• data_readers/ ← dataset loaders & augmentation
• Build/config files: CMakeLists.txt, setup.py, environment.yml
README
• Confirms core components: Python DPVO API, PyTorch nets, C++/CUDA acceleration, visualizer (Pangolin), classical loop closure (DBoW2 + DPRetrieval).
• Describes user workflows: demo → processing → optional SLAM back-end → DPViewer; train → network training; evaluate → batch evaluation
1. Entry-point layer (Python scripts: demo.py, train.py, evaluate_*.py)
2. Core DPVO pipeline (Python modules):
• DataReaders
• DeepNet (extractor.py + net.py)
• PatchCorr (altcorr C++/CUDA)
• FastBA (fastba C++/CUDA)
• LieTorch (C++/CUDA Lie algebra ops)
• LoopClosure logic (deep + optional classical)
3. External classical backend (optional):
• DBoW2 C++ library ←→ DPRetrieval pybind11
4. Visualization backend:
• DPViewer Python API ←→ pybind11 ←→ Pangolin C++ renderer
5. Persistent artifacts:
• Calibration/Config files
• Datasets (images + ground-truth)
• Pretrained models (download_models_and_data.sh)
• Logs & outputs (runs/, logs/, saved .ply/.txt)
• Scripts → load config + dataset → call DPVO API
• DPVO API sequentially:
DataReaders → DeepNet feature extraction → PatchCorr → FastBA optimization → LoopClosure
• LoopClosure → (if CLASSIC_LOOP_CLOSURE) call DPRetrieval → DBoW2 → back to LoopClosure
• Final poses/point-cloud → (if viz flag) → DPViewer → Pangolin display
• Training scripts → DPVO API + DeepNet → optimizer → save model → log
• Evaluation scripts → DPVO API → compute metrics → log/plot
c. Architectural patterns & principles
• Modular “processing pipeline” → each stage isolated (reader, network, corr, BA, closure)
• High-performance modules in C++/CUDA exposed via pybind11
• Optional classical extension (plug-in style)
• Python “orchestration layer” for ease of scripting and experiment management
d. Key technologies/frameworks
• Python 3.8+, PyTorch, numpy, ruamel.yaml
• C++17, CUDA kernels, Eigen
• pybind11 (for all C++↔Python bindings: altcorr, fastba, lietorch, DPRetrieval, DPViewer)
• Pangolin for 3D rendering
• OpenCV (optional classical backend)
• DBoW2 for bag-of-words retrieval