# shwplus/oa-vat architecture

> Architecture diagram of the GitHub repository shwplus/oa-vat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shwplus/oa-vat
- Repository: https://github.com/shwplus/oa-vat

## Overview

OA-VAT is an object-following system that combines target reference features, detection/tracking, and vehicle control; it also supports UnrealCV evaluation and optional diffusion-policy planning and person-identity enhancement. The primary workflow prepares a target feature file, initializes the runtime from `main.py`, processes video frames to track or reacquire the target, then sends motion commands. The README documents the feature-preparation and optional branches; sampled source confirms selected detector, worker, controller, and policy behavior, but not every runtime call between them.

Main entry points: [`main.py`](https://github.com/shwplus/oa-vat/blob/main/main.py) (Following runtime), [`extract_target_feature_dinov3.py`](https://github.com/shwplus/oa-vat/blob/main/extract_target_feature_dinov3.py) (Feature preparation), [`dp/dp_inference.py`](https://github.com/shwplus/oa-vat/blob/main/dp/dp_inference.py) (Policy inference), [`control/drone_control.py`](https://github.com/shwplus/oa-vat/blob/main/control/drone_control.py) (Vehicle control)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_features["Target features"]
  node_feature_extract["Feature preparation"]
  node_dinov3["DINOv3 encoder"]
  node_ipg["Person generation<br/>[inference.py]"]
  node_feature_file["Reference features"]
end

subgraph group_tracking["Tracking runtime"]
  node_main["Following runtime<br/>[main.py]"]
  node_yolo["YOLO detection"]
  node_ortrack["Object tracker<br/>[ortrack_tracker.py]"]
  node_workers["Enhance and reacquire<br/>[workers.py]"]
end

subgraph group_control["Control and planning"]
  node_dp["Policy inference<br/>[dp_inference.py]"]
  node_dp_model["Diffusion policy"]
  node_drone_control["Vehicle control<br/>[drone_control.py]"]
  node_sim_control["Simulation control"]
end

subgraph group_integrations["Optional integrations"]
  node_unreal_eval["Unreal evaluation"]
  node_unreal_env["UnrealCV environment<br/>[__init__.py]"]
end

node_user(("Operator"))
node_reference_images["Reference images"]
node_video["Video input"]
node_vehicle["Vehicle or simulator"]
node_video_output["Saved video"]

node_user -->|"prepares target"| node_feature_extract
node_reference_images -->|"supplies images"| node_feature_extract
node_feature_extract -->|"extracts features"| node_dinov3
node_feature_extract -.->|"optionally enhances"| node_ipg
node_feature_extract -->|"writes features"| node_feature_file
node_user -->|"starts following"| node_main
node_video -->|"provides frames"| node_main
node_feature_file -->|"loads target features"| node_main
node_main -->|"initializes detection"| node_yolo
node_main -->|"initializes tracking"| node_ortrack
node_main -.->|"optionally preloads"| node_dp
node_main -->|"runs enhancement"| node_workers
node_workers -->|"detects candidates"| node_yolo
node_main -->|"sends motion commands"| node_drone_control
node_main -.->|"controls simulation"| node_sim_control
node_dp -.->|"loads policy"| node_dp_model
node_dp -.->|"converts actions"| node_drone_control
node_drone_control -->|"commands vehicle"| node_vehicle
node_sim_control -.->|"steps environment"| node_unreal_env
node_unreal_eval -->|"runs episodes"| node_unreal_env
node_main -->|"writes recording"| node_video_output

click node_feature_extract "https://github.com/shwplus/oa-vat/blob/main/extract_target_feature_dinov3.py"
click node_dinov3 "https://github.com/shwplus/oa-vat/blob/main/dinov3_feature_extractor.py"
click node_ipg "https://github.com/shwplus/oa-vat/blob/main/Pose2ID/IPG/inference.py"
click node_main "https://github.com/shwplus/oa-vat/blob/main/main.py"
click node_yolo "https://github.com/shwplus/oa-vat/blob/main/ORTrack/inference_module.py"
click node_ortrack "https://github.com/shwplus/oa-vat/blob/main/detectors/ortrack_tracker.py"
click node_workers "https://github.com/shwplus/oa-vat/blob/main/tracking/workers.py"
click node_unreal_eval "https://github.com/shwplus/oa-vat/blob/main/eval_in_unrealcv.py"
click node_unreal_env "https://github.com/shwplus/oa-vat/blob/main/gym-unrealcv/gym_unrealcv/envs/tracking/__init__.py"
click node_dp "https://github.com/shwplus/oa-vat/blob/main/dp/dp_inference.py"
click node_dp_model "https://github.com/shwplus/oa-vat/tree/main/gym-unrealcv/diffusion_policy-main/diffusion_policy/policy"
click node_drone_control "https://github.com/shwplus/oa-vat/blob/main/control/drone_control.py"
click node_sim_control "https://github.com/shwplus/oa-vat/blob/main/DRONE/unreal_sim_controller.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_feature_extract,node_dinov3,node_ipg,node_feature_file toneBlue
class node_main,node_yolo,node_ortrack,node_workers toneAmber
class node_dp,node_dp_model,node_drone_control,node_sim_control toneMint
class node_unreal_eval,node_unreal_env toneRose
class node_user,node_reference_images,node_video,node_vehicle,node_video_output toneIndigo
```

## Components

### Target features

- [Feature preparation](https://github.com/shwplus/oa-vat/blob/main/extract_target_feature_dinov3.py): `extract_target_feature_dinov3.py`
- [DINOv3 encoder](https://github.com/shwplus/oa-vat/blob/main/dinov3_feature_extractor.py): `dinov3_feature_extractor.py`
- [Person generation](https://github.com/shwplus/oa-vat/blob/main/Pose2ID/IPG/inference.py): `Pose2ID/IPG/inference.py`
- **Reference features**

### Tracking runtime

- [Following runtime](https://github.com/shwplus/oa-vat/blob/main/main.py): `main.py`
- [YOLO detection](https://github.com/shwplus/oa-vat/blob/main/ORTrack/inference_module.py): `ORTrack/inference_module.py`
- [Object tracker](https://github.com/shwplus/oa-vat/blob/main/detectors/ortrack_tracker.py): `detectors/ortrack_tracker.py`
- [Enhance and reacquire](https://github.com/shwplus/oa-vat/blob/main/tracking/workers.py): `tracking/workers.py`

### Control and planning

- [Policy inference](https://github.com/shwplus/oa-vat/blob/main/dp/dp_inference.py): `dp/dp_inference.py`
- [Diffusion policy](https://github.com/shwplus/oa-vat/tree/main/gym-unrealcv/diffusion_policy-main/diffusion_policy/policy): `gym-unrealcv/diffusion_policy-main/diffusion_policy/policy`
- [Vehicle control](https://github.com/shwplus/oa-vat/blob/main/control/drone_control.py): `control/drone_control.py`
- [Simulation control](https://github.com/shwplus/oa-vat/blob/main/DRONE/unreal_sim_controller.py): `DRONE/unreal_sim_controller.py`

### Optional integrations

- [Unreal evaluation](https://github.com/shwplus/oa-vat/blob/main/eval_in_unrealcv.py): `eval_in_unrealcv.py`
- [UnrealCV environment](https://github.com/shwplus/oa-vat/blob/main/gym-unrealcv/gym_unrealcv/envs/tracking/__init__.py): `gym-unrealcv/gym_unrealcv/envs/tracking/__init__.py`

### Other components

- **Operator**
- **Reference images**
- **Video input**
- **Vehicle or simulator**
- **Saved video**

## Connections

- Operator → Feature preparation: prepares target
- Reference images → Feature preparation: supplies images
- Feature preparation → DINOv3 encoder: extracts features
- Feature preparation → Person generation: optionally enhances
- Feature preparation → Reference features: writes features
- Operator → Following runtime: starts following
- Video input → Following runtime: provides frames
- Reference features → Following runtime: loads target features
- Following runtime → YOLO detection: initializes detection
- Following runtime → Object tracker: initializes tracking
- Following runtime → Policy inference: optionally preloads
- Following runtime → Enhance and reacquire: runs enhancement
- Enhance and reacquire → YOLO detection: detects candidates
- Following runtime → Vehicle control: sends motion commands
- Following runtime → Simulation control: controls simulation
- Policy inference → Diffusion policy: loads policy
- Policy inference → Vehicle control: converts actions
- Vehicle control → Vehicle or simulator: commands vehicle
- Simulation control → UnrealCV environment: steps environment
- Unreal evaluation → UnrealCV environment: runs episodes
- Following runtime → Saved video: writes recording