# naver/dust3r architecture

> Architecture diagram of the GitHub repository naver/dust3r, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/naver/dust3r
- Repository: https://github.com/naver/dust3r

## Overview

##### Project type and purpose
- DUSt3R is a research-oriented 3D reconstruction framework (library + CLI + optional web UI).
- It ingests pairs of images, predicts per-pixel depth and confidence, refines a global 3D point cloud with an optimizer, and visualizes results.
- It includes data preprocessing scripts for multiple datasets, a PyTorch-based model, training and inference entrypoints, and Docker support for demo deployment.

- Raw images & metadata  → [Preprocessing layer] → Preprocessed dataset files
- Preprocessed files → [Dataset & Dataloader] → Batches of image pairs
- Batches → [Model Core] → Raw 3D predictions + confidences
- Predictions → [Global Alignment] → Refined poses + merged point cloud
- Refined output → [Visualization] → 3D viewer / evaluation metrics
- Training branch:
- Preprocessed files → [Dataset & Dataloader] → Batches + Labels
- Batches + Labels → [Model Core + Loss Engine] → Backprop → Checkpoints

##### Architectural patterns
- Modular layering: clear separation between data prep, data loading, model, training, inference, and deployment.
- Plugin-style datasets: each dataset implements a common base interface (base_stereo_view_dataset).
- Optimizer abstraction: `cloud_opt` packages define interchangeable alignment algorithms.
- Entrypoints (CLI scripts) glue modules into executable workflows.

##### Technologies & frameworks
- PyTorch (model, training, inference)
- Huggingface Hub (checkpoint retrieval)
- NumPy / SciPy (preprocessing, pair lists)
- Gradio (demo UI)
- Docker / Docker-Compose (containerized deployment)
- submodule: CroCo v2 for RoPE kernels (CUDA extensions)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        RawImages["Raw Images & Metadata"]:::external
        Preprocess["Data Preprocessing Layer"]:::dataBlue
        Preprocessed["Preprocessed Dataset"]:::dataBlue
    end
    RawImages -->|"raw images, metadata"| Preprocess
    Preprocess -->|"preprocessed files"| Preprocessed

    %% Data Loading Layer
    subgraph "Data Loading"
        Dataloader["Dataset & Dataloader Module"]:::dataBlue
    end
    Preprocessed -->|"batches of image pairs"| Dataloader

    %% Model Core
    subgraph "Model Core"
        ModelCore["Model Core"]:::mlGreen
    end
    Dataloader -->|"batches"| ModelCore

    %% Training Branch
    subgraph "Training Engine"
        LossEngine["Loss & Training Engine"]:::mlGreen
        Checkpoints["Checkpoints (.pth)"]:::mlGreen
    end
    ModelCore -->|"raw predictions"| LossEngine
    LossEngine -->|"checkpoints"| Checkpoints

    %% Inference & Alignment Branch
    subgraph "Inference & Global Alignment"
        Inference["Inference & Global Alignment"]:::mlGreen
        Refined["Refined Point Cloud + Poses"]:::mlGreen
    end
    ModelCore -->|"raw predictions"| Inference
    Inference -->|"point cloud + poses"| Refined

    %% Visualization & Evaluation
    subgraph "Visualization & Evaluation"
        Viz["Visualization & Evaluation"]:::orchesPurple
    end
    Refined -->|"visualization"| Viz

    %% Deployment
    subgraph "Deployment (Docker)"
        Docker["Docker Deployment"]:::orchesPurple
    end
    Docker --> Viz

    %% Click Events
    click Preprocess "https://github.com/naver/dust3r/tree/main/datasets_preprocess/"
    click Dataloader "https://github.com/naver/dust3r/tree/main/dust3r/datasets/"
    click ModelCore "https://github.com/naver/dust3r/blob/main/dust3r/model.py"
    click LossEngine "https://github.com/naver/dust3r/blob/main/train.py"
    click Inference "https://github.com/naver/dust3r/blob/main/dust3r/inference.py"
    click Viz "https://github.com/naver/dust3r/blob/main/demo.py"
    click Docker "https://github.com/naver/dust3r/tree/main/docker/"

    %% Styles
    classDef dataBlue fill:#ADD8E6,stroke:#333,stroke-width:1px;
    classDef mlGreen fill:#90EE90,stroke:#333,stroke-width:1px;
    classDef orchesPurple fill:#DDA0DD,stroke:#333,stroke-width:1px;
    classDef external fill:#F0E68C,stroke:#333,stroke-width:1px;
```
