# liheyoung/depth-anything architecture

> Architecture diagram of the GitHub repository liheyoung/depth-anything, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/liheyoung/depth-anything
- Repository: https://github.com/liheyoung/depth-anything

## Overview

##### Project Type and Purpose
- Depth Anything is a research-grade, full-stack machine-learning library and toolset for monocular depth estimation.
- It provides:
- Core relative-depth inference models (vision-transformer based)
- Fine-tuned metric-depth pipelines
- A depth-conditioned ControlNet module
- A semantic-segmentation downstream module
- CLI tools (`run.py`, `run_video.py`), a local Gradio demo (`app.py`), and integrations (HuggingFace Hub, ONNX, TensorRT).

##### File-Structure Analysis
- Top level:
- `app.py`: Gradio front end
- `run.py` / `run_video.py`: CLI inference scripts
- `depth_anything/`: core model implementation (encoders, transforms)
- `metric_depth/`: metric-depth training & evaluation pipelines
- `controlnet/`: improved depth-based ControlNet configs & code
- `semseg/`: semantic-segmentation configs & code
- `torchhub/`: packaging of Dinov2 for torch.hub
- `assets/`: sample inputs, outputs, paper, gallery
- Config files:
- JSON for ControlNet, semseg .py configs, environment.yml
- Build / deps:
- `requirements.txt`

##### Key README Insights
- Core “DepthAnything” class loads pre-trained ViT models via HF Hub or local weights
- Preprocessing pipeline uses `depth_anything.util.transform`
- Multiple downstream/extension points: metric_depth, controlnet, semseg
- External services: HuggingFace Spaces (demo), HF Hub (models), ONNX/TensorRT forks

##### Main Components
a. Input Layer
- Image Directory / Single Image / Text File / Video
- Asset Examples (`assets/examples`, `assets/examples_video`)
b. Preprocessing
- `depth_anything.util.transform` (Resize, Normalize, PrepareForNet)
- Configurable via CLI flags (`--grayscale`, `--pred-only`)
c. Core Model
- `DepthAnything` (from `depth_anything.dpt`)
- Three scales (Small/Base/Large) – loaded from HF Hub or locally
d. Postprocessing & Visualization
- Depth map coloring, side-by-side image+depth display
- Video frame stitching in `run_video.py`
- Gradio UI in `app.py`
e. Extension Modules
- Metric-Depth: fine-tuning & eval scripts (`metric_depth/`)
- ControlNet: retrained, config JSON, integration points (`controlnet/`)
- Semantic Segmentation: Mask2Former configs (`semseg/`)
f. External Services
- HuggingFace Hub (model checkpoints, Spaces demo)
- ONNX / TensorRT forks
- GitHub integrations (ControlNet WebUI, ComfyUI)

##### Relationships and Data Flows
- Inputs → Preprocessing → Core Model → Postprocessing → Outputs
- CLI tools and Gradio app consume the same preprocessing + model pipeline
- Extension modules tap into the core encoder for downstream tasks
- Config files steer ControlNet and semseg pipelines
- External services host or consume models (HF Hub, ONNX/TensorRT)

##### Architectural Patterns & Principles
- Modular ML Pipeline: clear separation between data I/O, transforms, model, and visualization
- Plugin-style extensions: metric_depth, controlnet, semseg each wrap the core encoder
- Config-driven: JSON and Python config files allow easy swapping of datasets/models
- CLI + GUI Dual Interface

##### Must-Include Elements
- Clear component names: “Preprocessing”, “DepthAnything Model”, “MetricDepth Trainer”, “ControlNet Plugin”, “SemSeg Plugin”
- Entry points: `run.py`, `run_video.py`, `app.py`

## Diagram (Mermaid)

```mermaid
graph TB
    %% Input Layer
    subgraph "Input Layer"
        direction TB
        Examples["Image Examples"]:::data
        Videos["Video Examples"]:::data
    end
    click Examples "https://github.com/liheyoung/depth-anything/tree/main/assets/examples"
    click Videos "https://github.com/liheyoung/depth-anything/tree/main/assets/examples_video"

    %% Preprocessing
    Preprocessing["Preprocessing\n(depth_anything/util/transform.py)"]:::code
    click Preprocessing "https://github.com/liheyoung/depth-anything/blob/main/depth_anything/util/transform.py"

    %% Core Model
    subgraph "Core Model"
        direction TB
        DPT["DepthAnything Model\n(depth_anything/dpt.py)"]:::code
        Blocks["Model Blocks\n(depth_anything/blocks.py)"]:::code
    end
    click DPT "https://github.com/liheyoung/depth-anything/blob/main/depth_anything/dpt.py"
    click Blocks "https://github.com/liheyoung/depth-anything/blob/main/depth_anything/blocks.py"

    %% Postprocessing & Visualization
    Postprocessing["Postprocessing & Visualization"]:::code

    %% Outputs
    Outputs["Outputs\n(Depth Maps, Videos)"]:::data

    %% Entry Points (CLI & GUI)
    subgraph "Entry Points"
        direction TB
        Run["run.py"]:::code
        RunVid["run_video.py"]:::code
        App["app.py"]:::code
    end
    click Run "https://github.com/liheyoung/depth-anything/blob/main/run.py"
    click RunVid "https://github.com/liheyoung/depth-anything/blob/main/run_video.py"
    click App "https://github.com/liheyoung/depth-anything/blob/main/app.py"

    %% Extension Modules
    subgraph "Extension Modules"
        direction TB
        Metric["MetricDepth Module"]:::code
        ControlNet["ControlNet Plugin"]:::code
        SemSeg["Semantic Segmentation Plugin"]:::code
        TorchHub["TorchHub Packaging"]:::code
    end
    click Metric "https://github.com/liheyoung/depth-anything/tree/main/metric_depth/"
    click ControlNet "https://github.com/liheyoung/depth-anything/blob/main/controlnet/config.json"
    click SemSeg "https://github.com/liheyoung/depth-anything/tree/main/semseg/config/depth_anything/"
    click TorchHub "https://github.com/liheyoung/depth-anything/tree/main/torchhub/"

    %% External Services
    subgraph "External Services"
        direction TB
        HFHub["HuggingFace Hub"]:::external
        ONNX["ONNX/TensorRT Forks"]:::external
        GitUIs["WebUIs\n(ControlNet WebUI, ComfyUI)"]:::external
    end

    %% Connections - Primary Pipeline
    Examples -->|"images"| Preprocessing
    Videos -->|"video frames"| Preprocessing
    Preprocessing -->|"torch.Tensor"| DPT
    Blocks --> DPT
    DPT -->|"raw depth"| Postprocessing
    Postprocessing -->|"visualizations"| Outputs

    %% Entry Points Connections
    Run --> Preprocessing
    RunVid --> Preprocessing
    RunVid --> Postprocessing
    App --> Preprocessing
    App --> Postprocessing

    %% Extension Dependencies (dashed)
    DPT -.-> Metric
    DPT -.-> ControlNet
    DPT -.-> SemSeg
    DPT -.-> TorchHub

    %% External Integrations
    DPT -->|"load weights"| HFHub
    Postprocessing -->|"export ONNX/TRT"| ONNX
    ControlNet -->|"WebUI integration"| GitUIs

    %% Legend
    subgraph "Legend"
        direction TB
        Code["Code Modules"]:::code
        Data["Data / Assets"]:::data
        External["External Services"]:::external
    end

    %% Styles
    classDef code fill:#D0E8FF,stroke:#0B5394,stroke-width:2px;
    classDef data fill:#D9EAD3,stroke:#38761D,stroke-width:2px,shape:cylinder;
    classDef external fill:#FFE6CC,stroke:#E69138,stroke-width:2px,shape:cloud;
```
