# saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection architecture

> Architecture diagram of the GitHub repository saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection
- Repository: https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection

## Overview

This repository implements QPIC human–object interaction detection, extended with hybrid-graph GNN candidate re-ranking and a Streamlit inference interface. The user workflow runs from image submission through QPIC detection, graph construction, GNN scoring, and ranked HOI results. Separate branches support dataset handling, model training, and evaluation, including HICO-DET and V-COCO. The README documents the principal product stages; source-index references support several internal calls. Some graph-generation and experiment scripts were not sampled, so their detailed links are left unasserted.

Main entry points: [`models/detr.py`](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/detr.py) (QPIC detector), [`predict_hoi.py`](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/predict_hoi.py) (HOI inference), [`gnn_model.py`](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/gnn_model.py) (HOI graph network), [`main.py`](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/main.py) (QPIC training entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["Inference interface"]
  node_streamlit["Streamlit app<br/>[app.py]"]
  node_adapter["HOI adapter<br/>[hoi_adapter.py]"]
end

subgraph group_reasoning["HOI reasoning"]
  node_predict["HOI inference<br/>[predict_hoi.py]"]
  node_qpic["QPIC detector<br/>[detr.py]"]
  node_graphbuilder["Hybrid graph builder"]
  node_gnn["HOI graph network<br/>[gnn_model.py]"]
  node_ranked["Ranked HOI results"]
end

subgraph group_data["Datasets and evaluation"]
  node_hico["HICO dataset<br/>[hico.py]"]
  node_coco["COCO dataset<br/>[coco.py]"]
  node_vcoco["V-COCO dataset<br/>[vcoco.py]"]
  node_transforms["Image transforms<br/>[transforms.py]"]
  node_evaluators["Dataset evaluators<br/>[hico_eval.py]"]
end

subgraph group_experiments["Training and evaluation"]
  node_engine["Training engine<br/>[engine.py]"]
  node_main["QPIC training entry<br/>[main.py]"]
  node_gnntrain["GNN training<br/>[train_gnn.py]"]
  node_gnneval["GNN evaluation<br/>[evaluate_gnn.py]"]
end

subgraph group_internals["Model internals"]
  node_backbone["Feature backbone<br/>[backbone.py]"]
  node_transformer["Transformer<br/>[transformer.py]"]
  node_hoihead["HOI head<br/>[hoi.py]"]
  node_segmentation["Segmentation head<br/>[segmentation.py]"]
  node_boxops["Box operations<br/>[box_ops.py]"]
  node_misc["Runtime utilities<br/>[misc.py]"]
end

node_user(("Image submitter"))

node_user -->|"submits image"| node_streamlit
node_streamlit -->|"uses adapter"| node_adapter
node_adapter -->|"invokes inference"| node_predict
node_adapter -->|"returns results"| node_streamlit
node_predict -->|"runs detector"| node_qpic
node_predict -->|"builds graph"| node_graphbuilder
node_predict -->|"scores candidates"| node_gnn
node_gnn -->|"refines ranking"| node_ranked
node_qpic -->|"uses features"| node_backbone
node_qpic -->|"uses transformer"| node_transformer
node_qpic -->|"uses HOI head"| node_hoihead
node_qpic -->|"uses masks"| node_segmentation
node_qpic -->|"uses box ops"| node_boxops
node_hico -->|"applies transforms"| node_transforms
node_coco -->|"applies transforms"| node_transforms
node_vcoco -->|"applies transforms"| node_transforms
node_main -->|"trains model"| node_engine
node_main -->|"loads dataset"| node_hico
node_main -->|"builds model"| node_qpic
node_engine -->|"evaluates outputs"| node_evaluators
node_engine -->|"logs metrics"| node_misc
node_gnntrain -->|"trains network"| node_gnn
node_gnneval -.->|"evaluates network"| node_gnn
node_hico -.->|"supports evaluation"| node_evaluators
node_vcoco -.->|"supports evaluation"| node_evaluators

click node_streamlit "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/Frontend_Streamlit/app.py"
click node_adapter "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/Frontend_Streamlit/hoi_adapter.py"
click node_predict "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/predict_hoi.py"
click node_qpic "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/detr.py"
click node_graphbuilder "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/build_hybrid_graph.py"
click node_gnn "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/gnn_model.py"
click node_hico "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/hico.py"
click node_coco "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/coco.py"
click node_vcoco "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/vcoco.py"
click node_transforms "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/transforms.py"
click node_evaluators "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/hico_eval.py"
click node_engine "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/engine.py"
click node_main "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/main.py"
click node_gnntrain "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/train_gnn.py"
click node_gnneval "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/evaluate_gnn.py"
click node_backbone "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/backbone.py"
click node_transformer "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/transformer.py"
click node_hoihead "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/hoi.py"
click node_segmentation "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/segmentation.py"
click node_boxops "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/util/box_ops.py"
click node_misc "https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/util/misc.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_streamlit,node_adapter toneBlue
class node_predict,node_qpic,node_graphbuilder,node_gnn,node_ranked toneAmber
class node_hico,node_coco,node_vcoco,node_transforms,node_evaluators toneMint
class node_engine,node_main,node_gnntrain,node_gnneval toneRose
class node_backbone,node_transformer,node_hoihead,node_segmentation,node_boxops,node_misc,node_user toneIndigo
```

## Components

### Inference interface

- [Streamlit app](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/Frontend_Streamlit/app.py): `Frontend_Streamlit/app.py`
- [HOI adapter](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/Frontend_Streamlit/hoi_adapter.py): `Frontend_Streamlit/hoi_adapter.py`

### HOI reasoning

- [HOI inference](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/predict_hoi.py): `predict_hoi.py`
- [QPIC detector](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/detr.py): `models/detr.py`
- [Hybrid graph builder](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/build_hybrid_graph.py): `build_hybrid_graph.py`
- [HOI graph network](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/gnn_model.py): `gnn_model.py`
- **Ranked HOI results**

### Datasets and evaluation

- [HICO dataset](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/hico.py): `datasets/hico.py`
- [COCO dataset](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/coco.py): `datasets/coco.py`
- [V-COCO dataset](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/vcoco.py): `datasets/vcoco.py`
- [Image transforms](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/transforms.py): `datasets/transforms.py`
- [Dataset evaluators](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/datasets/hico_eval.py): `datasets/hico_eval.py`

### Training and evaluation

- [Training engine](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/engine.py): `engine.py`
- [QPIC training entry](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/main.py): `main.py`
- [GNN training](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/train_gnn.py): `train_gnn.py`
- [GNN evaluation](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/evaluate_gnn.py): `evaluate_gnn.py`

### Model internals

- [Feature backbone](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/backbone.py): `models/backbone.py`
- [Transformer](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/transformer.py): `models/transformer.py`
- [HOI head](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/hoi.py): `models/hoi.py`
- [Segmentation head](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/models/segmentation.py): `models/segmentation.py`
- [Box operations](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/util/box_ops.py): `util/box_ops.py`
- [Runtime utilities](https://github.com/saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection/blob/main/util/misc.py): `util/misc.py`

### Other components

- **Image submitter**

## Connections

- Image submitter → Streamlit app: submits image (evidence: `README.md`)
- Streamlit app → HOI adapter: uses adapter (evidence: `Frontend_Streamlit/app.py`)
- HOI adapter → HOI inference: invokes inference (evidence: `Frontend_Streamlit/hoi_adapter.py`)
- HOI adapter → Streamlit app: returns results (evidence: `Frontend_Streamlit/app.py`)
- HOI inference → QPIC detector: runs detector (evidence: `predict_hoi.py`)
- HOI inference → Hybrid graph builder: builds graph (evidence: `predict_hoi.py`)
- HOI inference → HOI graph network: scores candidates (evidence: `predict_hoi.py`)
- HOI graph network → Ranked HOI results: refines ranking (evidence: `README.md`)
- QPIC detector → Feature backbone: uses features (evidence: `models/detr.py`)
- QPIC detector → Transformer: uses transformer (evidence: `models/detr.py`)
- QPIC detector → HOI head: uses HOI head (evidence: `models/detr.py`)
- QPIC detector → Segmentation head: uses masks (evidence: `models/detr.py`)
- QPIC detector → Box operations: uses box ops (evidence: `models/detr.py`)
- HICO dataset → Image transforms: applies transforms (evidence: `datasets/hico.py`)
- COCO dataset → Image transforms: applies transforms (evidence: `datasets/coco.py`)
- V-COCO dataset → Image transforms: applies transforms (evidence: `datasets/vcoco.py`)
- QPIC training entry → Training engine: trains model (evidence: `main.py`)
- QPIC training entry → HICO dataset: loads dataset (evidence: `main.py`)
- QPIC training entry → QPIC detector: builds model (evidence: `main.py`)
- Training engine → Dataset evaluators: evaluates outputs (evidence: `engine.py`)
- Training engine → Runtime utilities: logs metrics (evidence: `engine.py`)
- GNN training → HOI graph network: trains network (evidence: `train_gnn.py`)
- GNN evaluation → HOI graph network: evaluates network
- HICO dataset → Dataset evaluators: supports evaluation
- V-COCO dataset → Dataset evaluators: supports evaluation
