# princeton-vl/droid-slam architecture

> Architecture diagram of the GitHub repository princeton-vl/droid-slam, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/princeton-vl/droid-slam
- Repository: https://github.com/princeton-vl/droid-slam

## Overview

##### Project Type and Purpose
- DROID-SLAM is an open-source deep visual SLAM system for monocular, stereo, and RGB-D cameras. It combines deep neural network inference, classical geometry, and optimization to produce camera trajectories and dense reconstructions.
- It provides both inference (demo.py, view_reconstruction.py) and training (train.py) pipelines, plus evaluation scripts for standard SLAM benchmarks.

##### File-Structure-Based Component Analysis
Top-level directories and their responsibilities:
- droid_slam/
- Core SLAM pipeline: frontend, backend, optimization, network, geometry, modules, utilities
- src/
- Custom C++/CUDA extensions (kernels for correlation, feature matching)
- thirdparty/
- External libraries: lietorch, pytorch_scatter, tartanair_tools
- data/ and calib/
- Sample datasets and calibration files
- demo.py, train.py, view_reconstruction.py
- Entry points for inference, training, and visualization
- evaluation_scripts/
- Benchmark evaluation pipelines for ETH3D, EuRoC, TUM, TartanAir
- tools/
- Shell scripts to download data and pre-trained models
- requirements*.txt, environment*.yaml, setup.py
- Environment and install specifications

##### Key Architectural Patterns
- Pipeline separation between data ingestion, neural network inference (DROID Net), frontend processing, backend optimization (factor graph), and visualization.
- Asynchronous multiprocessing option: frontend and backend can run in separate processes/devices.
- Modular design: pluggable data_readers, network modules, geometry utilities, and visualization backends.
- Use of custom CUDA kernels for performance-critical operations.

##### Main Components and Their Interactions
a. Data Ingestion Layer
- droid_slam/data_readers: Reads images, depth, inertial data, applies augmentation.
- Inputs: raw sensor data + calibration files → Outputs: batched tensors.
b. DROID Network
- droid_slam/droid_net: Computes dense optical flow, feature correpondences.
- Uses custom CUDA kernels (src/*.cu) for correlation.
- Outputs: flow fields, correlation features.
c. Frontend
- droid_slam/droid_frontend / droid_slam/droid_async:
- Ingests network outputs, extracts keypoints (module: extractor), builds initial pose estimates.
d. Backend
- droid_slam/droid_backend:
- Builds factor graph (factor_graph.py), uses graph_utils & chol for optimization.
- Applies robust losses (losses.py) and solves for camera poses.
e. SLAM Core
- Motion filtering (motion_filter.py), trajectory filling (trajectory_filler.py).
- Align (align.py) for scale/alignment to ground truth.
f. Visualization
- droid_slam/visualizer: 3D scene and camera trajectory rendering via OpenGL (moderngl).
- view_reconstruction.py: High-resolution point cloud viewer.
g. Training
- train.py: Orchestrates data loading, forward pass through DROID net, backend, loss computation, backprop.
- Caches covisibility computations.
h. Benchmark Evaluation
- evaluation_scripts/*.py and tools/*.sh: Automate dataset download, inference, compute ATE/RPE, parse results.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion Layer
    subgraph "Data Ingestion Layer"
        direction TB
        RawData((Raw Sensor Data & Calibration)):::data
        DataReaders["Data Readers<br/>droid_slam/data_readers"]:::data
    end

    %% Neural Network & CUDA
    subgraph "Neural Network & CUDA Kernels"
        direction TB
        DROIDNet["DROID Net<br/>droid_slam/droid_net.py"]:::nn
        Modules["Network Modules<br/>droid_slam/modules"]:::nn
        CUDAKernels["CUDA Kernels<br/>src/*.cu"]:::nn
    end

    %% Frontend & Async
    subgraph "Frontend Processes"
        direction TB
        Frontend["Frontend<br/>droid_slam/droid_frontend.py"]:::proc
        AsyncFrontend["Async Frontend<br/>droid_slam/droid_async.py"]:::proc
    end

    %% Backend & Optimization
    subgraph "Backend & Optimization"
        direction TB
        Backend["Backend<br/>droid_slam/droid_backend.py"]:::opt
        FactorGraph["Factor Graph Optimizer<br/>droid_slam/factor_graph.py"]:::opt
        GeometryUtils["Geometry Utils<br/>droid_slam/geom"]:::opt
    end

    %% SLAM Core Utilities
    subgraph "SLAM Core Utilities"
        direction TB
        MotionFilter["Motion Filter<br/>droid_slam/motion_filter.py"]:::opt
        TrajectoryFiller["Trajectory Filler<br/>droid_slam/trajectory_filler.py"]:::opt
        Align["Align<br/>droid_slam/align.py"]:::opt
    end

    %% Visualization & Evaluation
    subgraph "Visualization & Evaluation"
        direction TB
        Visualizer["Visualizer<br/>droid_slam/visualizer"]:::viz
        Visualization["Visualization Utils<br/>droid_slam/visualization.py"]:::viz
        Demo["Demo Script<br/>demo.py"]:::script
        Train["Training Script<br/>train.py"]:::script
        ViewRec["Reconstruction Viewer<br/>view_reconstruction.py"]:::script
        EvalScripts["Evaluation Scripts<br/>evaluation_scripts"]:::script
    end

    %% External Libraries & Tools
    subgraph "External Dependencies"
        direction TB
        PyTorch(("PyTorch")):::ext
        LieTorch(("lietorch")):::ext
        Scatter(("pytorch_scatter")):::ext
        TartanTools(("tartanair_tools")):::ext
        Downloads["Download & Eval Tools<br/>tools/*.sh"]:::script
        DataCalib((Datasets & Calibration)):::data
    end

    %% Data Flow
    RawData -->|"images, depth, imu"| DataReaders
    DataReaders -->|"batched tensors"| Frontend

    %% Net Loop
    Frontend -->|"requests flow"| DROIDNet
    DROIDNet -->|"feature maps"| Modules
    Modules -->|"kernels call"| CUDAKernels
    CUDAKernels -->|"opt flow, corr"| DROIDNet
    DROIDNet -->|"outputs"| Frontend

    %% Async option
    Frontend -- IPC / queues --> AsyncFrontend
    AsyncFrontend --> Backend

    %% Backend Flow
    Backend -->|"build factors"| FactorGraph
    FactorGraph -->|"solve poses"| GeometryUtils
    GeometryUtils -->|"optimized poses"| Backend
    Backend -->|"poses"| MotionFilter
    MotionFilter --> TrajectoryFiller
    TrajectoryFiller --> Align
    Align --> Visualizer

    %% Scripts
    Demo --> Frontend
    Train --> DataReaders
    Train --> DROIDNet
    Train --> Backend
    Backend -->|"loss gradients"| Train
    ViewRec --> Visualizer
    EvalScripts --> Demo
    EvalScripts -->|parse results| EvalScripts
    Downloads --> EvalScripts

    %% External deps links
    DROIDNet --- PyTorch
    DROIDNet --- LieTorch
    Modules --- Scatter
    Modules --- TartanTools

    %% Click Events
    click DataReaders "https://github.com/princeton-vl/droid-slam/tree/main/droid_slam/data_readers"
    click DROIDNet "https://github.com/princeton-vl/droid-slam/blob/main/droid_slam/droid_net.py"
    click Modules "https://github.com/princeton-vl/droid-slam/tree/main/droid_slam/modules"
    click CUDAKernels "https://github.com/princeton-vl/droid-slam/tree/main/src"
    click Frontend "https://github.com/princeton-vl/droid-slam/blob/main/droid_slam/droid_frontend.py"
    click AsyncFrontend "https://github.com/princeton-vl/droid-slam/blob/main/droid_slam/droid_async.py"
    click Backend "https://github.com/princeton-vl/droid-slam/blob/main/droid_slam/droid_backend.py"
    click FactorGraph "https://github.com/princeton-vl/droid-slam/blob/main/droid_slam/factor_graph.py"
    click GeometryUtils "https://github.com/princeton-vl/droid-slam/tree/main/droid_slam/geom"
    click MotionFilter "https://github.com/princeton-vl/droid-slam/blob/main/droid_slam/motion_filter.py"
    click TrajectoryFiller "https://github.com/princeton-vl/droid-slam/blob/main/droid_slam/trajectory_filler.py"
    click Align "https://github.com/princeton-vl/droid-slam/blob/main/droid_slam/align.py"
    click Visualizer "https://github.com/princeton-vl/droid-slam/tree/main/droid_slam/visualizer"
    click Visualization "https://github.com/princeton-vl/droid-slam/blob/main/droid_slam/visualization.py"
    click Demo "https://github.com/princeton-vl/droid-slam/blob/main/demo.py"
    click Train "https://github.com/princeton-vl/droid-slam/blob/main/train.py"
    click ViewRec "https://github.com/princeton-vl/droid-slam/blob/main/view_reconstruction.py"
    click EvalScripts "https://github.com/princeton-vl/droid-slam/tree/main/evaluation_scripts"
    click Downloads "https://github.com/princeton-vl/droid-slam/tree/main/tools"

    %% Legend Styles
    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef nn fill:#d4edda,stroke:#333,stroke-width:1px
    classDef proc fill:#cfe2ff,stroke:#333,stroke-width:1px
    classDef opt fill:#fff3cd,stroke:#333,stroke-width:1px
    classDef viz fill:#e2e3e5,stroke:#333,stroke-width:1px
    classDef script fill:#f5c6cb,stroke:#333,stroke-width:1px
    classDef ext fill:#f8f9fa,stroke:#333,stroke-width:1px
```
