# depthanything/depth-anything-v2 architecture

> Architecture diagram of the GitHub repository depthanything/depth-anything-v2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/depthanything/depth-anything-v2
- Repository: https://github.com/depthanything/depth-anything-v2

## Overview

##### Project type and purpose
- This is a research‐oriented, Python-based depth estimation library and toolkit (“Depth Anything V2”) with both inference and training pipelines.
- It provides:
- A core relative depth model (depth_anything_v2) based on DINOv2+DPT
- A metric depth fine‐tuning extension (metric_depth)
- Command‐line scripts for image/video inference (run.py, run_video.py)
- A Gradio demo server (app.py)

##### Main architectural components
A. Core Library
- depth_anything_v2/ – model definition (DINOv2 encoder, DPT decoder) and utility modules (transform, blocks)
- checkpoints/ (external) – pre‐trained model weights downloaded from Hugging Face

B. Metric Depth Extension
- metric_depth/depth_anything_v2/ – mirrored core code for metric training
- metric_depth/dataset/ – data loaders and splits for Hypersim, KITTI, VKITTI2
- metric_depth/util/ – loss, metrics, distributed helpers
- metric_depth/train.py, run.py, dist_train.sh – training and evaluation orchestration

C. Inference & Demo
- run.py & run_video.py (root) – CLI wrappers that:
– parse args → load model from checkpoints → preprocess via depth_anything_v2/util → infer → postprocess & save
- app.py – Gradio application
- assets/ – example images/videos

D. External Integrations
- PyTorch (model, training, inference)
- OpenCV/Pillow (I/O, preprocessing)
- Hugging Face Transformers (optional inference pipeline)
- Gradio (web demo)
- Apple Core ML, ONNX, TensorRT (community ports)

##### Relationships & data flows
- Training flow:
metric_depth/train.py → uses metric_depth/dataset → depth_anything_v2 model code → metric_depth/util/loss & metric → optimizer step → checkpoint write
(distributed via dist_train.sh)
- Inference flow (images/videos):
- External model fetch: scripts download from checkpoints or load via Transformers pipeline

##### Architectural patterns & design principles
- Modular layering: separate “Model Definition,” “Data I/O,” “Training Utils,” “Inference CLI,” and “Web Serving”
- Code reuse: metric_depth re-uses core depth_anything_v2 code under its own namespace for fine‐tuning
- Pipeline abstraction: CLI and Gradio wrap the same core inference logic
- Configuration over code: encoder types and hyperparameters passed via command‐line or dict

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Core Model Library"
        encoder["DINOv2 Encoder"]:::core
        decoder["DPT Decoder"]:::core
        blocks["blocks.py"]:::core
        transform["transform.py"]:::core
        postproc["Postprocess Utilities"]:::core
    end

    subgraph "Metric Depth Module"
        subgraph "Mirrored Core"
            m_encoder["DINOv2 Encoder"]:::core
            m_decoder["DPT Decoder"]:::core
            m_blocks["blocks.py"]:::core
            m_transform["transform.py"]:::core
        end
        subgraph "Dataset Loaders"
            hypersim["hypersim.py"]:::core
            kitti["kitti.py"]:::core
            vkitti2["vkitti2.py"]:::core
            ds_transform["transform.py"]:::core
            splits["splits/"]:::core
        end
        subgraph "Training Scripts"
            trainScript["train.py"]:::script
            runScript["run.py"]:::script
            distScript["dist_train.sh"]:::script
        end
        subgraph "Training Utils"
            loss["loss.py"]:::core
            metricCal["metric.py"]:::core
            distHelper["dist_helper.py"]:::core
            utils["utils.py"]:::core
        end
    end

    subgraph "Inference CLI"
        cli_img["run.py"]:::script
        cli_vid["run_video.py"]:::script
    end

    subgraph "Gradio Web Demo"
        web["app.py"]:::script
        assets["assets/"]:::script
        examples_video["assets/examples_video/"]:::script
    end

    subgraph "External Services"
        hf["Checkpoints (Hugging Face)"]:::ext
        pytorch["PyTorch"]:::ext
        opencv["OpenCV/Pillow"]:::ext
        transformers["HF Transformers"]:::ext_dashed
        gradio_lib["Gradio"]:::ext
        coreml["Core ML"]:::ext_dashed
        onnx["ONNX"]:::ext_dashed
        tensorrt["TensorRT"]:::ext_dashed
    end

    %% Training Flow
    hypersim -->|loads data| trainScript
    kitti -->|loads data| trainScript
    vkitti2 -->|loads data| trainScript
    ds_transform -->|preprocess| trainScript
    splits -->|splits info| trainScript
    trainScript -->|Python import| m_encoder
    trainScript -->|Python import| m_decoder
    trainScript -->|use| loss
    trainScript -->|use| metricCal
    trainScript -->|use| distHelper
    trainScript -->|use| utils
    trainScript -->|PyTorch forward/backward| pytorch
    trainScript -->|Checkpoint save| hf

    %% Inference Flow
    cli_img -->|Raw Image| transform
    cli_img -->|Python import| encoder
    encoder -->|features| decoder
    decoder -->|feature map| postproc
    postproc -->|Depth Map| cli_img
    cli_img -->|Checkpoint load| hf

    cli_vid -->|Raw Video| transform
    cli_vid -->|Python import| encoder
    cli_vid -->|features| decoder
    cli_vid -->|feature map| postproc
    cli_vid -->|Depth Video| cli_vid
    cli_vid -->|Checkpoint load| hf

    %% Web Demo Flow
    web -->|HTTP REST| transform
    web -->|Python import| encoder
    web -->|features| decoder
    web -->|feature map| postproc
    web -->|Depth Map| web
    web -->|Checkpoint load| hf
    web -->|serves| assets
    web -->|serves| examples_video

    %% External Integrations
    encoder -->|backend| pytorch
    decoder -->|backend| pytorch
    transform -->|backend| opencv
    postproc -->|backend| opencv
    web -->|backend| gradio_lib
    transformers -.->|optional| encoder
    coreml -.->|community| encoder
    onnx -.->|community| encoder
    tensorrt -.->|community| encoder

    %% Metric Core Reuse
    m_encoder -->|mirror| encoder
    m_decoder -->|mirror| decoder
    m_blocks -->|mirror| blocks
    m_transform -->|mirror| transform

    %% Styles
    classDef core fill:#D6EAF8,stroke:#3498DB
    classDef script fill:#D5F5E3,stroke:#27AE60
    classDef ext fill:#FCF3CF,stroke:#F1C40F
    classDef ext_dashed stroke:#F39C12,stroke-dasharray:5 5

    %% Click Events
    click encoder "https://github.com/depthanything/depth-anything-v2/blob/main/depth_anything_v2/dinov2.py"
    click decoder "https://github.com/depthanything/depth-anything-v2/blob/main/depth_anything_v2/dpt.py"
    click blocks "https://github.com/depthanything/depth-anything-v2/blob/main/depth_anything_v2/util/blocks.py"
    click transform "https://github.com/depthanything/depth-anything-v2/blob/main/depth_anything_v2/util/transform.py"
    click m_encoder "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/depth_anything_v2/dinov2.py"
    click m_decoder "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/depth_anything_v2/dpt.py"
    click m_blocks "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/depth_anything_v2/util/blocks.py"
    click m_transform "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/depth_anything_v2/util/transform.py"
    click hypersim "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/dataset/hypersim.py"
    click kitti "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/dataset/kitti.py"
    click vkitti2 "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/dataset/vkitti2.py"
    click ds_transform "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/dataset/transform.py"
    click splits "https://github.com/depthanything/depth-anything-v2/tree/main/metric_depth/dataset/splits/"
    click trainScript "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/train.py"
    click runScript "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/run.py"
    click distScript "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/dist_train.sh"
    click loss "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/util/loss.py"
    click metricCal "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/util/metric.py"
    click distHelper "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/util/dist_helper.py"
    click utils "https://github.com/depthanything/depth-anything-v2/blob/main/metric_depth/util/utils.py"
    click cli_img "https://github.com/depthanything/depth-anything-v2/blob/main/run.py"
    click cli_vid "https://github.com/depthanything/depth-anything-v2/blob/main/run_video.py"
    click web "https://github.com/depthanything/depth-anything-v2/blob/main/app.py"
    click assets "https://github.com/depthanything/depth-anything-v2/tree/main/assets/"
    click examples_video "https://github.com/depthanything/depth-anything-v2/tree/main/assets/examples_video/"
```
