# vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection architecture

> Architecture diagram of the GitHub repository vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection
- Repository: https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection

## Overview

This repository combines the OVT-B open-vocabulary multi-object-tracking benchmark and an OVTrack baseline with a separate Diffusers-based image-generation package. The main benchmark workflow is to configure OVT-B class prompts and data paths, load video annotations, run a tracker, and evaluate tracking results. The tree also contains model training and multiple tracker implementations. Diffusion pipelines and their model, scheduler, and configuration layers are included as a distinct runtime branch; source evidence for several connections is limited to the SOURCE INDEX or README.

Main entry points: [`diffusers_clean/pipelines/stable_diffusion/pipeline_ovtrack_image_generation.py`](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/diffusers_clean/pipelines/stable_diffusion/pipeline_ovtrack_image_generation.py) (OVTrack image pipeline), [`ovtrack/models/mot/ovtrack.py`](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/models/mot/ovtrack.py) (OVTrack model), [`ovtrack/models/trackers`](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/ovtrack/models/trackers) (Tracker family), [`diffusers_clean/schedulers`](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/diffusers_clean/schedulers) (Diffusion schedulers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_benchmark["OVT-B benchmark"]
  node_ovtb_data[("OVT-B data")]
  node_class_names["OVT-B classes<br/>[ovtb_classname.py]"]
  node_tracking_configs["Tracking configs<br/>[ovtrack_r50_ua.py]"]
  node_video_dataset["Video datasets"]
  node_coco_parser["Video annotation parser"]
  node_evaluation["MOT evaluation<br/>[mot.py]"]
end

subgraph group_tracking["OVTrack runtime"]
  node_inference_cli["Inference entry<br/>[inference.py]"]
  node_inference_api["Inference API<br/>[inference.py]"]
  node_model_registry["Model builder<br/>[builder.py]"]
  node_ovtrack_model["OVTrack model<br/>[ovtrack.py]"]
  node_roi_heads["Detection and tracking heads"]
  node_tracker_family["Tracker family"]
  node_association["Track association<br/>[association.py]"]
  node_kalman["Adaptive motion filter<br/>[adaptive_kalman.py]"]
  node_training_api["Training API<br/>[train.py]"]
end

subgraph group_diffusion["Diffusion generation"]
  node_image_generation["OVTrack image pipeline"]
  node_ovtrack_diffusion["Tracking-aware diffusion"]
end

subgraph group_diffusion_core["Diffusion components"]
  node_diffusion_models["Denoising and VAE models"]
  node_diffusion_schedulers["Diffusion schedulers"]
  node_pipeline_base["Pipeline base<br/>[pipeline_utils.py]"]
  node_configuration["Model configuration"]
end

node_researcher(("Researcher"))

node_researcher -->|"downloads"| node_ovtb_data
node_researcher -->|"configures classes"| node_class_names
node_researcher -->|"sets data paths"| node_tracking_configs
node_ovtb_data -.->|"provides annotations"| node_video_dataset
node_video_dataset -->|"uses parser"| node_coco_parser
node_researcher -->|"runs tracking"| node_inference_cli
node_inference_cli -->|"calls inference"| node_inference_api
node_inference_api -->|"builds model"| node_model_registry
node_model_registry -.->|"registers model"| node_ovtrack_model
node_ovtrack_model -.->|"uses heads"| node_roi_heads
node_ovtrack_model -.->|"dispatches tracking"| node_tracker_family
node_tracker_family -.->|"associates tracks"| node_association
node_tracker_family -.->|"uses motion filter"| node_kalman
node_researcher -->|"evaluates results"| node_evaluation
node_researcher -.->|"trains baseline"| node_training_api
node_image_generation -->|"uses models"| node_diffusion_models
node_image_generation -->|"uses scheduler"| node_diffusion_schedulers
node_image_generation -->|"registers components"| node_pipeline_base
node_ovtrack_diffusion -->|"uses models"| node_diffusion_models
node_ovtrack_diffusion -->|"uses scheduler"| node_diffusion_schedulers
node_pipeline_base -->|"inherits configuration"| node_configuration
node_diffusion_schedulers -->|"uses configuration"| node_configuration

click node_class_names "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/data/ovtb/ovtb_classname.py"
click node_tracking_configs "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/configs/ovtrack-teta/ovtb/ovtrack_r50_ua.py"
click node_video_dataset "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/datasets/coco_video_dataset.py"
click node_coco_parser "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/datasets/parsers/coco_video_parser.py"
click node_inference_cli "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/tools/inference.py"
click node_inference_api "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/apis/inference.py"
click node_model_registry "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/models/builder.py"
click node_ovtrack_model "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/models/mot/ovtrack.py"
click node_roi_heads "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/ovtrack/models/roi_heads"
click node_tracker_family "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/ovtrack/models/trackers"
click node_association "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/core/track/association.py"
click node_kalman "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/models/trackers/adaptive_kalman.py"
click node_evaluation "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/core/evaluation/mot.py"
click node_training_api "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/apis/train.py"
click node_image_generation "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/diffusers_clean/pipelines/stable_diffusion/pipeline_ovtrack_image_generation.py"
click node_ovtrack_diffusion "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/diffusers_clean/pipelines/stable_diffusion/pipeline_stable_diffusion_ovtrack.py"
click node_diffusion_models "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/diffusers_clean/models"
click node_diffusion_schedulers "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/diffusers_clean/schedulers"
click node_pipeline_base "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/diffusers_clean/pipeline_utils.py"
click node_configuration "https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/diffusers_clean/configuration_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ovtb_data,node_class_names,node_tracking_configs,node_video_dataset,node_coco_parser,node_evaluation toneBlue
class node_inference_cli,node_inference_api,node_model_registry,node_ovtrack_model,node_roi_heads,node_tracker_family,node_association,node_kalman,node_training_api toneAmber
class node_image_generation,node_ovtrack_diffusion toneMint
class node_diffusion_models,node_diffusion_schedulers,node_pipeline_base,node_configuration toneRose
class node_researcher toneIndigo
```

## Components

### OVT-B benchmark

- **OVT-B data**
- [OVT-B classes](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/data/ovtb/ovtb_classname.py): `data/ovtb/ovtb_classname.py`
- [Tracking configs](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/configs/ovtrack-teta/ovtb/ovtrack_r50_ua.py): `configs/ovtrack-teta/ovtb/ovtrack_r50_ua.py`
- [Video datasets](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/datasets/coco_video_dataset.py): `ovtrack/datasets/coco_video_dataset.py`
- [Video annotation parser](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/datasets/parsers/coco_video_parser.py): `ovtrack/datasets/parsers/coco_video_parser.py`
- [MOT evaluation](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/core/evaluation/mot.py): `ovtrack/core/evaluation/mot.py`

### OVTrack runtime

- [Inference entry](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/tools/inference.py): `tools/inference.py`
- [Inference API](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/apis/inference.py): `ovtrack/apis/inference.py`
- [Model builder](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/models/builder.py): `ovtrack/models/builder.py`
- [OVTrack model](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/models/mot/ovtrack.py): `ovtrack/models/mot/ovtrack.py`
- [Detection and tracking heads](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/ovtrack/models/roi_heads): `ovtrack/models/roi_heads`
- [Tracker family](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/ovtrack/models/trackers): `ovtrack/models/trackers`
- [Track association](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/core/track/association.py): `ovtrack/core/track/association.py`
- [Adaptive motion filter](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/models/trackers/adaptive_kalman.py): `ovtrack/models/trackers/adaptive_kalman.py`
- [Training API](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/ovtrack/apis/train.py): `ovtrack/apis/train.py`

### Diffusion generation

- [OVTrack image pipeline](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/diffusers_clean/pipelines/stable_diffusion/pipeline_ovtrack_image_generation.py): `diffusers_clean/pipelines/stable_diffusion/pipeline_ovtrack_image_generation.py`
- [Tracking-aware diffusion](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/diffusers_clean/pipelines/stable_diffusion/pipeline_stable_diffusion_ovtrack.py): `diffusers_clean/pipelines/stable_diffusion/pipeline_stable_diffusion_ovtrack.py`

### Diffusion components

- [Denoising and VAE models](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/diffusers_clean/models): `diffusers_clean/models`
- [Diffusion schedulers](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/tree/main/diffusers_clean/schedulers): `diffusers_clean/schedulers`
- [Pipeline base](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/diffusers_clean/pipeline_utils.py): `diffusers_clean/pipeline_utils.py`
- [Model configuration](https://github.com/vamsi419-cell/uncertanity-adpative-kalam-fusion-for-open-vocabulary-multi-object-detection/blob/main/diffusers_clean/configuration_utils.py): `diffusers_clean/configuration_utils.py`

### Other components

- **Researcher**

## Connections

- Researcher → OVT-B data: downloads (evidence: `README.md`)
- Researcher → OVT-B classes: configures classes (evidence: `README.md`)
- Researcher → Tracking configs: sets data paths (evidence: `README.md`)
- OVT-B data → Video datasets: provides annotations
- Video datasets → Video annotation parser: uses parser (evidence: `ovtrack/datasets/coco_video_dataset.py`)
- Researcher → Inference entry: runs tracking (evidence: `README.md`)
- Inference entry → Inference API: calls inference (evidence: `tools/inference.py`)
- Inference API → Model builder: builds model (evidence: `ovtrack/apis/inference.py`)
- Model builder → OVTrack model: registers model
- OVTrack model → Detection and tracking heads: uses heads
- OVTrack model → Tracker family: dispatches tracking
- Tracker family → Track association: associates tracks
- Tracker family → Adaptive motion filter: uses motion filter
- Researcher → MOT evaluation: evaluates results (evidence: `README.md`)
- Researcher → Training API: trains baseline
- OVTrack image pipeline → Denoising and VAE models: uses models (evidence: `diffusers_clean/pipelines/stable_diffusion/pipeline_ovtrack_image_generation.py`)
- OVTrack image pipeline → Diffusion schedulers: uses scheduler (evidence: `diffusers_clean/pipelines/stable_diffusion/pipeline_ovtrack_image_generation.py`)
- OVTrack image pipeline → Pipeline base: registers components (evidence: `diffusers_clean/pipelines/stable_diffusion/pipeline_ovtrack_image_generation.py`)
- Tracking-aware diffusion → Denoising and VAE models: uses models (evidence: `diffusers_clean/pipelines/stable_diffusion/pipeline_stable_diffusion_ovtrack.py`)
- Tracking-aware diffusion → Diffusion schedulers: uses scheduler (evidence: `diffusers_clean/pipelines/stable_diffusion/pipeline_stable_diffusion_ovtrack.py`)
- Pipeline base → Model configuration: inherits configuration (evidence: `diffusers_clean/pipeline_utils.py`)
- Diffusion schedulers → Model configuration: uses configuration (evidence: `diffusers_clean/schedulers/scheduling_pndm_flax.py`)
