How niconielsen32/yolo-3d works
File Structure
• Top-level scripts:
• run.py: orchestrates the entire pipeline, parses configuration & CLI.
• detection_model.py: wraps YOLOv11 inference.
• depth_model.py: wraps Depth Anything v2 inference.
• load_camera_params.py: handles retrieval and caching of camera intrinsics/extrinsics.
• requirements.txt: lists external dependencies (PyTorch, OpenCV, NumPy, etc.).
• No front-end directory—pure CLI/Python application.
README for Insights
• Feature toggles: tracking, BEV, pseudo-3D overlays.
• Supported inputs: video file or live webcam.
• Automatic model weights download on first run.
Main Components & Their Interactions
a. Input Layer
• Source selector in run.py: Video file reader (OpenCV) or Webcam capture.
b. Inference Layer
1. Detection Service (detection_model.py)
2. Depth Service (depth_model.py)
• Takes same frame → returns depth map (per-pixel depth).
c. Fusion Layer
• bbox3d_utils.py
d. Visualization & Tracking Layer
• Optional tracker (could be internal to run.py or detection_model).
• 3D overlay renderer (OpenCV drawing).
e. Output Layer
• Frame writer (OpenCV VideoWriter) or on-screen display.
• Optionally saves to file via path configured in run.py.
Architectural Patterns & Design Principles
• Pipeline pattern: sequential, stage-based processing of video frames.
• Single orchestration script (run.py) with pluggable modules for each stage.
• Separation of concerns: detection, depth estimation, fusion, rendering each in its own module.
• Configurable feature toggles promote open/closed principle—able to extend visualization modes without rewriting core loop.
Relevant Technologies & Frameworks
• PyTorch: deep‐learning inference for YOLOv11 & Depth Anything v2.
• OpenCV: video I/O, frame capture, drawing utilities.
• NumPy: numerical operations and tensor manipulation.
• Optional tracker library (if enabled).
• Python 3.8+, pip-managed dependencies via requirements.txt.
6. Branch from fusion to optional Tracker module and Visualization module (BEV + 3D overlay).
8. A side note for “model weights auto-download” feeding into both model services.