How fantasybarry/bev works
BEV combines monocular depth, 3D object detection, and driving-scene segmentation to turn image or video input into depth-aware bird’s-eye-view (BEV) outputs, including GPS-referenced overlays. The main workflow is orchestrated by OmniLineDepth.py; the README also documents real-time processing, batch processing, and video reconstruction. The graph includes the sampled depth, projection, and Omni3D internals, plus the documented YOLOPv2 and GPS branches. Some integration details and unsampled subsystem internals are not shown, so only documented or indexed relationships are connected.
Main entry points: OmniLineDepth.py Inference orchestrator · rcnn3d.py 3D detector · roi_heads.py 3D ROI heads · visual.py BEV projection
Components 20
BEV pipeline
- Inference orchestrator
OmniLineDepth.py - Real-time runner
realTimeOmni.py - BEV and video output
OmniLineDepth.py
Depth estimation
- Metric depth model
DepthAnythingV2/metric_depth/depth_anything_v2/dpt.py - DINOv2 encoder
DepthAnythingV2/metric_depth/depth_anything_v2/dinov2.py - Transformer layers
DepthAnythingV2/metric_depth/depth_anything_v2/dinov2_layers/block.py
3D detection
- Omni3D datasets
omni3d/cubercnn/data/datasets.py - DLA backbone
omni3d/cubercnn/modeling/backbone/dla.py - 3D detector
omni3d/cubercnn/modeling/meta_arch/rcnn3d.py - Region proposals
omni3d/cubercnn/modeling/proposal_generator/rpn.py - 3D ROI heads
omni3d/cubercnn/modeling/roi_heads/roi_heads.py - Box prediction
omni3d/cubercnn/modeling/roi_heads/fast_rcnn.py
Driving perception
- Driving perception
YOLOPv2/utils/utils.py
Projection and overlays
- Depth inference tools
projection/tools.py - BEV projection
projection/visual.py - Projection tools
projection/tools.py - 3D scene renderer
omni3d/cubercnn/vis/vis.py - GPS integration
overlay/gps/update.py - Overlay app
overlay/app.py
Other components
- Video / image input
Connections 21
- Video / image input → Inference orchestrator: provides frames
README.md - Inference orchestrator → Metric depth model: estimates depth
OmniLineDepth.py - Inference orchestrator → 3D detector: runs detection
OmniLineDepth.py - Inference orchestrator → Driving perception: runs perception
OmniLineDepth.py - Inference orchestrator → BEV projection: projects masks
OmniLineDepth.py - Inference orchestrator → BEV and video output: writes video
OmniLineDepth.py - Inference orchestrator → Real-time runner: supports real-time
README.md - Metric depth model → DINOv2 encoder: extracts features
DepthAnythingV2/metric_depth/depth_anything_v2/dpt.py - DINOv2 encoder → Transformer layers: builds transformer
DepthAnythingV2/metric_depth/depth_anything_v2/dinov2.py - Projection tools → Metric depth model: loads model
projection/tools.py - BEV projection → Projection tools: uses depth tools
projection/visual.py - BEV projection → 3D scene renderer: renders scene
projection/visual.py - 3D detector → DLA backbone: uses backbone
- 3D detector → Region proposals: generates proposals
- 3D detector → 3D ROI heads: predicts instances
- 3D ROI heads → Region proposals: uses proposals
omni3d/cubercnn/modeling/roi_heads/roi_heads.py - 3D ROI heads → Box prediction: uses box head
omni3d/cubercnn/modeling/roi_heads/roi_heads.py - Omni3D datasets → 3D detector: provides training data
- Inference orchestrator → GPS integration: integrates GPS
README.md - GPS integration → Overlay app: georeferences output
README.md - Overlay app → BEV and video output: composes overlay
README.md