# naver/mast3r architecture

> Architecture diagram of the GitHub repository naver/mast3r, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/naver/mast3r
- Repository: https://github.com/naver/mast3r

## Overview

Based on our analysis of the file tree and README, this project is a computer vision system for 3D image matching and reconstruction – specifically, the “MASt3R” framework for grounding image matching in 3D. It is designed to perform matching between images and reconstruct 3D structures, and it even integrates dual pipelines (e.g., MASt3R and DUSt3R) for both metric regression and retrieval tasks. The project includes demos, training scripts, and docker support, which makes it both an experimentation platform and an interactive demo environment.

##### Main Components
- Data Handling & Preprocessing:
– Directories: mast3r/datasets, mast3r/datasets/base, mast3r/datasets/utils.
– Includes utilities for cropping, handling stereo view datasets, and preparing inputs.
- Model Definition & Inference Engine:
– Core code inside mast3r/model.py and head implementations (e.g., mast3r/catmlp_dpt_head.py).
– Contains model architecture (e.g., AsymmetricMASt3R) and configurations like patch embeddings and transformer layers.
- Matching & Retrieval:
– Retrieval module in mast3r/retrieval, including graph and processor modules.
– Uses fast nearest neighbors from mast3r/fast_nn.py and additional matching components from dust3r/inference.
- 3D Reconstruction & Optimization:
– The “cloud_opt” directory inside mast3r includes functionality for triangulation, TSDF optimization, and sparse global alignment.
- Demo & Visualization:
– Demo scripts (demo.py, demo_dust3r_ga.py, demo_glomap.py) provide interactive systems and visualization of matching results.
- Docker & Deployment:
– The docker folder with Dockerfiles, docker-compose configurations, and related entrypoint scripts for containerized demos.
- External Integration:
– Submodules (e.g., asmk) and additional requirements (e.g., installation of cythonized CUDA kernels in dust3r) that support retrieval and faster numeric computations.

##### Relationships and Data Flow
– Input images and datasets are first ingested by the Data Handling modules.
– Preprocessing and dataset utilities feed data to the Model Definition components.
– The inference engine (using AsymmetricMASt3R) processes images; descriptors are generated and then provided to the Matching & Retrieval modules.
– The matching engine (using fast_reciprocal_NNs and retrieval models) calculates correspondences between image pairs.
– Further, reconstruction or refinement modules (e.g., cloud optimization and triangulation) work on these matches to generate 3D data.
– Final outputs are visualized via demo scripts and can also be deployed via docker containers for interactive demos.

##### Architectural Patterns and Technologies
– Modular design: Different directories indicate a separation of concerns (datasets, model, utilities, retrieval, and visualization).
– Pipeline architecture: Data flows sequentially from input to pre-processing, model inference, matching, and finally visualization, with clear intermediate representations.
– Docker containerization: Separate docker-compose files and deployment scripts indicate the use of containerization for reproducible environments (both CPU and CUDA enabled runs).
– Deep Learning and 3D Vision: Involvement of PyTorch, convolutional and transformer architectures, and CUDA kernels for performance.
– Integration of external libraries and submodules (e.g., asmk for ASMK features extraction).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Handling & Preprocessing"
        DS["Datasets"]
        DSB["Datasets Base"]
        DSU["Datasets Utils"]
    end

    subgraph "Model Definition & Inference Engine"
        MD["model.py"]
        HC["catmlp_dpt_head.py"]
    end

    subgraph "Matching & Retrieval"
        MR["Retrieval Module"]
        NN["fast_nn.py"]
    end

    subgraph "3D Reconstruction & Optimization"
        RE["Cloud Optimization"]
        Spa["Sparse GA"]
        Tri["Triangulation"]
        TSDF["TSDF Optimizer"]
    end

    subgraph "Demo & Visualization"
        D1["demo.py"]
        D2["demo_dust3r_ga.py"]
        D3["demo_glomap.py"]
        D4["mast3r/demo.py"]
        D5["mast3r/demo_glomap.py"]
        VZ["visloc.py"]
    end

    subgraph "Docker & Deployment"
        DC["docker"]
        DCPU["docker-compose-cpu.yml"]
        DCUDA["docker-compose-cuda.yml"]
        DF["docker/files"]
        DRS["docker/run.sh"]
    end

    subgraph "External Integration"
        EX["dust3r"]
    end

    %% Data flow connections
    DS --> MD
    DSB --> MD
    DSU --> MD
    MD --> HC
    MD --> MR
    HC --> MR
    MR --> NN
    MR --> RE
    RE --> Spa
    RE --> Tri
    RE --> TSDF
    RE --> D1
    D1 --> D2
    D2 --> D3
    D3 --> D4
    D4 --> D5
    D5 --> VZ
    D1 --> DC
    EX --> MR

    %% Styles
    classDef datasets fill:#F8C471,stroke:#333,stroke-width:2px;
    classDef model fill:#85C1E9,stroke:#333,stroke-width:2px;
    classDef retrieval fill:#82E0AA,stroke:#333,stroke-width:2px;
    classDef recon fill:#F9E79F,stroke:#333,stroke-width:2px;
    classDef demo fill:#F5CBA7,stroke:#333,stroke-width:2px;
    classDef docker fill:#E6B0AA,stroke:#333,stroke-width:2px;
    classDef external fill:#D7BDE2,stroke:#333,stroke-width:2px;

    class DS,DSB,DSU datasets;
    class MD,HC model;
    class MR,NN retrieval;
    class RE,Spa,Tri,TSDF recon;
    class D1,D2,D3,D4,D5,VZ demo;
    class DC,DCPU,DCUDA,DF,DRS docker;
    class EX external;

    %% Click Events for Data Handling & Preprocessing
    click DS "https://github.com/naver/mast3r/tree/main/mast3r/datasets"
    click DSB "https://github.com/naver/mast3r/tree/main/mast3r/datasets/base"
    click DSU "https://github.com/naver/mast3r/tree/main/mast3r/datasets/utils"

    %% Click Events for Model Definition & Inference Engine
    click MD "https://github.com/naver/mast3r/blob/main/mast3r/model.py"
    click HC "https://github.com/naver/mast3r/blob/main/mast3r/catmlp_dpt_head.py"

    %% Click Events for Matching & Retrieval
    click MR "https://github.com/naver/mast3r/tree/main/mast3r/retrieval"
    click NN "https://github.com/naver/mast3r/blob/main/mast3r/fast_nn.py"

    %% Click Events for 3D Reconstruction & Optimization
    click RE "https://github.com/naver/mast3r/tree/main/mast3r/cloud_opt"
    click Spa "https://github.com/naver/mast3r/blob/main/mast3r/cloud_opt/sparse_ga.py"
    click Tri "https://github.com/naver/mast3r/blob/main/mast3r/cloud_opt/triangulation.py"
    click TSDF "https://github.com/naver/mast3r/blob/main/mast3r/cloud_opt/tsdf_optimizer.py"

    %% Click Events for Demo & Visualization
    click D1 "https://github.com/naver/mast3r/blob/main/demo.py"
    click D2 "https://github.com/naver/mast3r/blob/main/demo_dust3r_ga.py"
    click D3 "https://github.com/naver/mast3r/blob/main/demo_glomap.py"
    click D4 "https://github.com/naver/mast3r/blob/main/mast3r/demo.py"
    click D5 "https://github.com/naver/mast3r/blob/main/mast3r/demo_glomap.py"
    click VZ "https://github.com/naver/mast3r/blob/main/visloc.py"

    %% Click Events for Docker & Deployment
    click DC "https://github.com/naver/mast3r/tree/main/docker"
    click DCPU "https://github.com/naver/mast3r/blob/main/docker/docker-compose-cpu.yml"
    click DCUDA "https://github.com/naver/mast3r/blob/main/docker/docker-compose-cuda.yml"
    click DF "https://github.com/naver/mast3r/tree/main/docker/files"
    click DRS "https://github.com/naver/mast3r/blob/main/docker/run.sh"

    %% Click Events for External Integration
    click EX "https://github.com/naver/mast3r/tree/main/dust3r"
```
