# zzz512/promptdepth architecture

> Architecture diagram of the GitHub repository zzz512/promptdepth, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zzz512/promptdepth
- Repository: https://github.com/zzz512/promptdepth

## Overview

PromptDepth is a promptable monocular-depth system with stereo/video depth, interactive prompts, and mask-guided tracking, plus a training pipeline. The graph follows the README’s documented workflows and sampled model/data code; several entrypoint implementations were not sampled, so their documented connections are shown without finer internal call details.

Main entry points: [`prompt_depth/dpt.py`](https://github.com/zzz512/promptdepth/blob/main/prompt_depth/dpt.py) (PromptDepth model), [`video_infer.py`](https://github.com/zzz512/promptdepth/blob/main/video_infer.py) (Stereo/video inference), [`video_prompt_infer.py`](https://github.com/zzz512/promptdepth/blob/main/video_prompt_infer.py) (Mask-guided tracking), [`inference.py`](https://github.com/zzz512/promptdepth/blob/main/inference.py) (Image inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflows["User workflows"]
  node_image_infer["Image inference<br/>[inference.py]"]
  node_video_infer["Stereo/video inference<br/>[video_infer.py]"]
  node_prompt_video["Mask-guided tracking"]
  node_demo["Interactive demo<br/>[demo.py]"]
  node_trt["TensorRT inference<br/>[trt_inference.py]"]
end

subgraph group_training["Training data"]
  node_train["Training entrypoint<br/>[train.py]"]
  node_dataset["Paired-frame dataset<br/>[dataset.py]"]
  node_data_prep["Dataset preparation"]
  node_transforms["Data transforms<br/>[transform.py]"]
end

subgraph group_model["Depth model"]
  node_model["PromptDepth model<br/>[dpt.py]"]
  node_dinov2["DINOv2 encoder<br/>[dinov2.py]"]
  node_prompt_encoder["Prompt encoder<br/>[prompt_encoder.py]"]
  node_transformer["Prompt transformer<br/>[transformer.py]"]
end

subgraph group_video_tools["Video utilities"]
  node_visualize["Depth visualization"]
  node_render["Masked video rendering"]
end

node_user(("User"))
node_weights["Model checkpoint"]
node_outputs["Depth outputs"]
node_frames["Video frames and masks"]

node_user -->|"runs image inference"| node_image_infer
node_user -->|"runs video depth"| node_video_infer
node_user -->|"starts tracking"| node_prompt_video
node_user -->|"opens demo"| node_demo
node_image_infer -->|"loads checkpoint"| node_weights
node_video_infer -->|"loads checkpoint"| node_weights
node_prompt_video -->|"loads checkpoint"| node_weights
node_demo -->|"loads checkpoint"| node_weights
node_image_infer -->|"predicts depth"| node_model
node_video_infer -->|"predicts paired depth"| node_model
node_prompt_video -->|"predicts prompted depth"| node_model
node_demo -->|"serves predictions"| node_model
node_image_infer -->|"writes depth"| node_outputs
node_video_infer -->|"writes depth sequence"| node_outputs
node_prompt_video -->|"writes prompted sequence"| node_outputs
node_user -->|"starts training"| node_train
node_data_prep -->|"generates training lists"| node_dataset
node_train -->|"loads training pairs"| node_dataset
node_dataset -->|"transforms samples"| node_transforms
node_train -->|"trains model"| node_model
node_model -->|"encodes images"| node_dinov2
node_model -->|"encodes prompts"| node_prompt_encoder
node_model -->|"fuses prompt features"| node_transformer
node_user -.->|"runs optimized inference"| node_trt
node_trt -.->|"returns depth"| node_outputs
node_user -->|"visualizes depth sequence"| node_visualize
node_outputs -->|"reads depth array"| node_visualize
node_visualize -->|"writes color frames"| node_outputs
node_user -->|"renders masked video"| node_render
node_outputs -->|"reads depth frames"| node_render
node_frames -->|"supplies masks"| node_render
node_render -->|"writes masked video"| node_outputs
node_frames -->|"supplies adjacent frames"| node_video_infer
node_frames -->|"supplies frames and masks"| node_prompt_video

click node_image_infer "https://github.com/zzz512/promptdepth/blob/main/inference.py"
click node_video_infer "https://github.com/zzz512/promptdepth/blob/main/video_infer.py"
click node_prompt_video "https://github.com/zzz512/promptdepth/blob/main/video_prompt_infer.py"
click node_demo "https://github.com/zzz512/promptdepth/blob/main/demo.py"
click node_train "https://github.com/zzz512/promptdepth/blob/main/train.py"
click node_dataset "https://github.com/zzz512/promptdepth/blob/main/dataset/dataset.py"
click node_data_prep "https://github.com/zzz512/promptdepth/tree/main/dataset"
click node_transforms "https://github.com/zzz512/promptdepth/blob/main/dataset/transform.py"
click node_model "https://github.com/zzz512/promptdepth/blob/main/prompt_depth/dpt.py"
click node_dinov2 "https://github.com/zzz512/promptdepth/blob/main/prompt_depth/dinov2.py"
click node_prompt_encoder "https://github.com/zzz512/promptdepth/blob/main/prompt_depth/util/prompt_encoder.py"
click node_transformer "https://github.com/zzz512/promptdepth/blob/main/prompt_depth/util/transformer.py"
click node_trt "https://github.com/zzz512/promptdepth/blob/main/trt_inference.py"
click node_visualize "https://github.com/zzz512/promptdepth/blob/main/tools/visualize_depth_video.py"
click node_render "https://github.com/zzz512/promptdepth/blob/main/tools/create_prompt_video.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_image_infer,node_video_infer,node_prompt_video,node_demo,node_trt,node_user toneBlue
class node_train,node_dataset,node_data_prep,node_transforms toneAmber
class node_model,node_dinov2,node_prompt_encoder,node_transformer toneMint
class node_visualize,node_render toneRose
class node_weights,node_outputs,node_frames toneIndigo
```

## Components

### User workflows

- [Image inference](https://github.com/zzz512/promptdepth/blob/main/inference.py): `inference.py`
- [Stereo/video inference](https://github.com/zzz512/promptdepth/blob/main/video_infer.py): `video_infer.py`
- [Mask-guided tracking](https://github.com/zzz512/promptdepth/blob/main/video_prompt_infer.py): `video_prompt_infer.py`
- [Interactive demo](https://github.com/zzz512/promptdepth/blob/main/demo.py): `demo.py`
- [TensorRT inference](https://github.com/zzz512/promptdepth/blob/main/trt_inference.py): `trt_inference.py`

### Training data

- [Training entrypoint](https://github.com/zzz512/promptdepth/blob/main/train.py): `train.py`
- [Paired-frame dataset](https://github.com/zzz512/promptdepth/blob/main/dataset/dataset.py): `dataset/dataset.py`
- [Dataset preparation](https://github.com/zzz512/promptdepth/tree/main/dataset): `dataset`
- [Data transforms](https://github.com/zzz512/promptdepth/blob/main/dataset/transform.py): `dataset/transform.py`

### Depth model

- [PromptDepth model](https://github.com/zzz512/promptdepth/blob/main/prompt_depth/dpt.py): `prompt_depth/dpt.py`
- [DINOv2 encoder](https://github.com/zzz512/promptdepth/blob/main/prompt_depth/dinov2.py): `prompt_depth/dinov2.py`
- [Prompt encoder](https://github.com/zzz512/promptdepth/blob/main/prompt_depth/util/prompt_encoder.py): `prompt_depth/util/prompt_encoder.py`
- [Prompt transformer](https://github.com/zzz512/promptdepth/blob/main/prompt_depth/util/transformer.py): `prompt_depth/util/transformer.py`

### Video utilities

- [Depth visualization](https://github.com/zzz512/promptdepth/blob/main/tools/visualize_depth_video.py): `tools/visualize_depth_video.py`
- [Masked video rendering](https://github.com/zzz512/promptdepth/blob/main/tools/create_prompt_video.py): `tools/create_prompt_video.py`

### Other components

- **User**
- **Model checkpoint**
- **Depth outputs**
- **Video frames and masks**

## Connections

- User → Image inference: runs image inference
- User → Stereo/video inference: runs video depth
- User → Mask-guided tracking: starts tracking
- User → Interactive demo: opens demo
- Image inference → Model checkpoint: loads checkpoint
- Stereo/video inference → Model checkpoint: loads checkpoint
- Mask-guided tracking → Model checkpoint: loads checkpoint
- Interactive demo → Model checkpoint: loads checkpoint
- Image inference → PromptDepth model: predicts depth
- Stereo/video inference → PromptDepth model: predicts paired depth
- Mask-guided tracking → PromptDepth model: predicts prompted depth
- Interactive demo → PromptDepth model: serves predictions
- Image inference → Depth outputs: writes depth
- Stereo/video inference → Depth outputs: writes depth sequence
- Mask-guided tracking → Depth outputs: writes prompted sequence
- User → Training entrypoint: starts training
- Dataset preparation → Paired-frame dataset: generates training lists
- Training entrypoint → Paired-frame dataset: loads training pairs
- Paired-frame dataset → Data transforms: transforms samples
- Training entrypoint → PromptDepth model: trains model
- PromptDepth model → DINOv2 encoder: encodes images
- PromptDepth model → Prompt encoder: encodes prompts
- PromptDepth model → Prompt transformer: fuses prompt features
- User → TensorRT inference: runs optimized inference
- TensorRT inference → Depth outputs: returns depth
- User → Depth visualization: visualizes depth sequence
- Depth outputs → Depth visualization: reads depth array
- Depth visualization → Depth outputs: writes color frames
- User → Masked video rendering: renders masked video
- Depth outputs → Masked video rendering: reads depth frames
- Video frames and masks → Masked video rendering: supplies masks
- Masked video rendering → Depth outputs: writes masked video
- Video frames and masks → Stereo/video inference: supplies adjacent frames
- Video frames and masks → Mask-guided tracking: supplies frames and masks