How vnk8071/depth-anything-triton works
Project Type and Purpose
• This is a machine-learning inference toolkit and simple UI that:
• Converts a PyTorch depth-estimation model to ONNX and TensorRT formats
• Provides both CLI-based and Triton-based inference
• Hosts a minimal web UI for interactive use
File Structure
• Top-level scripts:
• export.py – exports PyTorch → ONNX
• onnx2trt.py – compiles ONNX → TensorRT engine
• infer.py – CLI ONNX/TensorRT inference
• depth_anything_triton_infer.py – CLI Triton server client
• app.py – launches local web UI
• Deployment artifacts:
• Dockerfile & compose.yml – build/run Triton server container
• model_repository/… – Triton model repository layout with config.pbtxt
• Assets & support:
• assets/ – sample images, videos, UI screenshots
• utils.py – shared helper functions
• requirements.txt – Python dependencies
README Insights
• Conversion workflows (ONNX, TensorRT)
• CLI usage flags and model inputs/outputs
• Web UI serving on port 7860
• Docker/Triton deployment steps and performance benchmarking
Main Components
a. Model Conversion Pipeline
– PyTorch checkpoint → export.py → ONNX model
– onnx2trt.py or trtexec → TensorRT engine
b. Local Inference Layer
– infer.py (direct ONNX/engine)
– app.py (UI calls infer.py under the hood)
c. Triton Inference Server
– Docker/compose.yml containerizing Triton
– model_repository/depth_anything
– config.pbtxt + model.plan
d. Triton Client
– depth_anything_triton_infer.py
– HTTP/gRPC calls to Triton API
e. External Inputs/Outputs
– User uploads images/videos (CLI flags or UI)
– Depth maps returned (files or HTTP responses)
Relationships and Data Flow
• Conversion scripts produce model artifacts stored in local folders (models/, model_repository/).
• CLI inference and UI both load from these artifacts when --mode onnx/trt is selected.
• Triton container mounts model_repository and exposes gRPC/HTTP endpoints.
• Triton client script calls those endpoints and retrieves depth results.
• Web UI (app.py) may call local inference functions directly or send requests to Triton if configured.
Architectural Patterns and Technologies
• Pattern: modular pipeline (conversion → inference → UI/deployment)
• Technologies: Python, ONNX, TensorRT, NVIDIA Triton Inference Server, Docker, Flask/FastAPI (likely behind app.py), polygraphy/trtexec, perf_analyzer