# fantasybarry/bev architecture

> Architecture diagram of the GitHub repository fantasybarry/bev, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fantasybarry/bev
- Repository: https://github.com/fantasybarry/bev

## Overview

BEV combines monocular depth, 3D object detection, and driving-scene segmentation to turn image or video input into depth-aware bird’s-eye-view (BEV) outputs, including GPS-referenced overlays. The main workflow is orchestrated by OmniLineDepth.py; the README also documents real-time processing, batch processing, and video reconstruction. The graph includes the sampled depth, projection, and Omni3D internals, plus the documented YOLOPv2 and GPS branches. Some integration details and unsampled subsystem internals are not shown, so only documented or indexed relationships are connected.

Main entry points: [`OmniLineDepth.py`](https://github.com/fantasybarry/bev/blob/main/OmniLineDepth.py) (Inference orchestrator), [`omni3d/cubercnn/modeling/meta_arch/rcnn3d.py`](https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/meta_arch/rcnn3d.py) (3D detector), [`omni3d/cubercnn/modeling/roi_heads/roi_heads.py`](https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/roi_heads/roi_heads.py) (3D ROI heads), [`projection/visual.py`](https://github.com/fantasybarry/bev/blob/main/projection/visual.py) (BEV projection)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["BEV pipeline"]
  node_pipeline["Inference orchestrator<br/>[OmniLineDepth.py]"]
  node_realtime["Real-time runner<br/>[realTimeOmni.py]"]
  node_output["BEV and video output<br/>[OmniLineDepth.py]"]
end

subgraph group_depth["Depth estimation"]
  node_depthmodel["Metric depth model<br/>[dpt.py]"]
  node_dinov2["DINOv2 encoder<br/>[dinov2.py]"]
  node_dinolayers["Transformer layers<br/>[block.py]"]
end

subgraph group_omni["3D detection"]
  node_datasets["Omni3D datasets<br/>[datasets.py]"]
  node_backbone["DLA backbone<br/>[dla.py]"]
  node_detector["3D detector<br/>[rcnn3d.py]"]
  node_rpn["Region proposals<br/>[rpn.py]"]
  node_roi["3D ROI heads<br/>[roi_heads.py]"]
  node_fast_rcnn["Box prediction<br/>[fast_rcnn.py]"]
end

subgraph group_driving["Driving perception"]
  node_yolo["Driving perception<br/>[utils.py]"]
end

subgraph group_mapping["Projection and overlays"]
  node_depthutils["Depth inference tools<br/>[tools.py]"]
  node_projection["BEV projection<br/>[visual.py]"]
  node_geometry["Projection tools<br/>[tools.py]"]
  node_renderer["3D scene renderer<br/>[vis.py]"]
  node_gps["GPS integration<br/>[update.py]"]
  node_overlay["Overlay app<br/>[app.py]"]
end

node_user(("Video / image input"))

node_user -->|"provides frames"| node_pipeline
node_pipeline -->|"estimates depth"| node_depthmodel
node_pipeline -->|"runs detection"| node_detector
node_pipeline -->|"runs perception"| node_yolo
node_pipeline -->|"projects masks"| node_projection
node_pipeline -->|"writes video"| node_output
node_pipeline -.->|"supports real-time"| node_realtime
node_depthmodel -->|"extracts features"| node_dinov2
node_dinov2 -->|"builds transformer"| node_dinolayers
node_geometry -->|"loads model"| node_depthmodel
node_projection -->|"uses depth tools"| node_geometry
node_projection -->|"renders scene"| node_renderer
node_detector -.->|"uses backbone"| node_backbone
node_detector -.->|"generates proposals"| node_rpn
node_detector -.->|"predicts instances"| node_roi
node_roi -->|"uses proposals"| node_rpn
node_roi -->|"uses box head"| node_fast_rcnn
node_datasets -.->|"provides training data"| node_detector
node_pipeline -.->|"integrates GPS"| node_gps
node_gps -.->|"georeferences output"| node_overlay
node_overlay -.->|"composes overlay"| node_output

click node_pipeline "https://github.com/fantasybarry/bev/blob/main/OmniLineDepth.py"
click node_realtime "https://github.com/fantasybarry/bev/blob/main/realTimeOmni.py"
click node_depthmodel "https://github.com/fantasybarry/bev/blob/main/DepthAnythingV2/metric_depth/depth_anything_v2/dpt.py"
click node_dinov2 "https://github.com/fantasybarry/bev/blob/main/DepthAnythingV2/metric_depth/depth_anything_v2/dinov2.py"
click node_dinolayers "https://github.com/fantasybarry/bev/blob/main/DepthAnythingV2/metric_depth/depth_anything_v2/dinov2_layers/block.py"
click node_depthutils "https://github.com/fantasybarry/bev/blob/main/projection/tools.py"
click node_datasets "https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/data/datasets.py"
click node_backbone "https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/backbone/dla.py"
click node_detector "https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/meta_arch/rcnn3d.py"
click node_rpn "https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/proposal_generator/rpn.py"
click node_roi "https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/roi_heads/roi_heads.py"
click node_fast_rcnn "https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/roi_heads/fast_rcnn.py"
click node_yolo "https://github.com/fantasybarry/bev/blob/main/YOLOPv2/utils/utils.py"
click node_projection "https://github.com/fantasybarry/bev/blob/main/projection/visual.py"
click node_geometry "https://github.com/fantasybarry/bev/blob/main/projection/tools.py"
click node_renderer "https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/vis/vis.py"
click node_gps "https://github.com/fantasybarry/bev/blob/main/overlay/gps/update.py"
click node_overlay "https://github.com/fantasybarry/bev/blob/main/overlay/app.py"
click node_output "https://github.com/fantasybarry/bev/blob/main/OmniLineDepth.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline,node_realtime,node_output toneBlue
class node_depthmodel,node_dinov2,node_dinolayers toneAmber
class node_datasets,node_backbone,node_detector,node_rpn,node_roi,node_fast_rcnn toneMint
class node_yolo toneRose
class node_depthutils,node_projection,node_geometry,node_renderer,node_gps,node_overlay,node_user toneIndigo
```

## Components

### BEV pipeline

- [Inference orchestrator](https://github.com/fantasybarry/bev/blob/main/OmniLineDepth.py): `OmniLineDepth.py`
- [Real-time runner](https://github.com/fantasybarry/bev/blob/main/realTimeOmni.py): `realTimeOmni.py`
- [BEV and video output](https://github.com/fantasybarry/bev/blob/main/OmniLineDepth.py): `OmniLineDepth.py`

### Depth estimation

- [Metric depth model](https://github.com/fantasybarry/bev/blob/main/DepthAnythingV2/metric_depth/depth_anything_v2/dpt.py): `DepthAnythingV2/metric_depth/depth_anything_v2/dpt.py`
- [DINOv2 encoder](https://github.com/fantasybarry/bev/blob/main/DepthAnythingV2/metric_depth/depth_anything_v2/dinov2.py): `DepthAnythingV2/metric_depth/depth_anything_v2/dinov2.py`
- [Transformer layers](https://github.com/fantasybarry/bev/blob/main/DepthAnythingV2/metric_depth/depth_anything_v2/dinov2_layers/block.py): `DepthAnythingV2/metric_depth/depth_anything_v2/dinov2_layers/block.py`

### 3D detection

- [Omni3D datasets](https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/data/datasets.py): `omni3d/cubercnn/data/datasets.py`
- [DLA backbone](https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/backbone/dla.py): `omni3d/cubercnn/modeling/backbone/dla.py`
- [3D detector](https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/meta_arch/rcnn3d.py): `omni3d/cubercnn/modeling/meta_arch/rcnn3d.py`
- [Region proposals](https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/proposal_generator/rpn.py): `omni3d/cubercnn/modeling/proposal_generator/rpn.py`
- [3D ROI heads](https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/roi_heads/roi_heads.py): `omni3d/cubercnn/modeling/roi_heads/roi_heads.py`
- [Box prediction](https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/modeling/roi_heads/fast_rcnn.py): `omni3d/cubercnn/modeling/roi_heads/fast_rcnn.py`

### Driving perception

- [Driving perception](https://github.com/fantasybarry/bev/blob/main/YOLOPv2/utils/utils.py): `YOLOPv2/utils/utils.py`

### Projection and overlays

- [Depth inference tools](https://github.com/fantasybarry/bev/blob/main/projection/tools.py): `projection/tools.py`
- [BEV projection](https://github.com/fantasybarry/bev/blob/main/projection/visual.py): `projection/visual.py`
- [Projection tools](https://github.com/fantasybarry/bev/blob/main/projection/tools.py): `projection/tools.py`
- [3D scene renderer](https://github.com/fantasybarry/bev/blob/main/omni3d/cubercnn/vis/vis.py): `omni3d/cubercnn/vis/vis.py`
- [GPS integration](https://github.com/fantasybarry/bev/blob/main/overlay/gps/update.py): `overlay/gps/update.py`
- [Overlay app](https://github.com/fantasybarry/bev/blob/main/overlay/app.py): `overlay/app.py`

### Other components

- **Video / image input**

## Connections

- Video / image input → Inference orchestrator: provides frames (evidence: `README.md`)
- Inference orchestrator → Metric depth model: estimates depth (evidence: `OmniLineDepth.py`)
- Inference orchestrator → 3D detector: runs detection (evidence: `OmniLineDepth.py`)
- Inference orchestrator → Driving perception: runs perception (evidence: `OmniLineDepth.py`)
- Inference orchestrator → BEV projection: projects masks (evidence: `OmniLineDepth.py`)
- Inference orchestrator → BEV and video output: writes video (evidence: `OmniLineDepth.py`)
- Inference orchestrator → Real-time runner: supports real-time (evidence: `README.md`)
- Metric depth model → DINOv2 encoder: extracts features (evidence: `DepthAnythingV2/metric_depth/depth_anything_v2/dpt.py`)
- DINOv2 encoder → Transformer layers: builds transformer (evidence: `DepthAnythingV2/metric_depth/depth_anything_v2/dinov2.py`)
- Projection tools → Metric depth model: loads model (evidence: `projection/tools.py`)
- BEV projection → Projection tools: uses depth tools (evidence: `projection/visual.py`)
- BEV projection → 3D scene renderer: renders scene (evidence: `projection/visual.py`)
- 3D detector → DLA backbone: uses backbone
- 3D detector → Region proposals: generates proposals
- 3D detector → 3D ROI heads: predicts instances
- 3D ROI heads → Region proposals: uses proposals (evidence: `omni3d/cubercnn/modeling/roi_heads/roi_heads.py`)
- 3D ROI heads → Box prediction: uses box head (evidence: `omni3d/cubercnn/modeling/roi_heads/roi_heads.py`)
- Omni3D datasets → 3D detector: provides training data
- Inference orchestrator → GPS integration: integrates GPS (evidence: `README.md`)
- GPS integration → Overlay app: georeferences output (evidence: `README.md`)
- Overlay app → BEV and video output: composes overlay (evidence: `README.md`)
