How saiprasad0905/dynamic-knowledge-guided-reasoning-for-human-object-interaction-detection works
This repository implements QPIC human–object interaction detection, extended with hybrid-graph GNN candidate re-ranking and a Streamlit inference interface. The user workflow runs from image submission through QPIC detection, graph construction, GNN scoring, and ranked HOI results. Separate branches support dataset handling, model training, and evaluation, including HICO-DET and V-COCO. The README documents the principal product stages; source-index references support several internal calls. Some graph-generation and experiment scripts were not sampled, so their detailed links are left unasserted.
Main entry points: detr.py QPIC detector · predict_hoi.py HOI inference · gnn_model.py HOI graph network · main.py QPIC training entry
Components 23
Inference interface
- Streamlit app
Frontend_Streamlit/app.py - HOI adapter
Frontend_Streamlit/hoi_adapter.py
HOI reasoning
- HOI inference
predict_hoi.py - QPIC detector
models/detr.py - Hybrid graph builder
build_hybrid_graph.py - HOI graph network
gnn_model.py - Ranked HOI results
Datasets and evaluation
- HICO dataset
datasets/hico.py - COCO dataset
datasets/coco.py - V-COCO dataset
datasets/vcoco.py - Image transforms
datasets/transforms.py - Dataset evaluators
datasets/hico_eval.py
Training and evaluation
- Training engine
engine.py - QPIC training entry
main.py - GNN training
train_gnn.py - GNN evaluation
evaluate_gnn.py
Model internals
- Feature backbone
models/backbone.py - Transformer
models/transformer.py - HOI head
models/hoi.py - Segmentation head
models/segmentation.py - Box operations
util/box_ops.py - Runtime utilities
util/misc.py
Other components
- Image submitter
Connections 25
- Image submitter → Streamlit app: submits image
README.md - Streamlit app → HOI adapter: uses adapter
Frontend_Streamlit/app.py - HOI adapter → HOI inference: invokes inference
Frontend_Streamlit/hoi_adapter.py - HOI adapter → Streamlit app: returns results
Frontend_Streamlit/app.py - HOI inference → QPIC detector: runs detector
predict_hoi.py - HOI inference → Hybrid graph builder: builds graph
predict_hoi.py - HOI inference → HOI graph network: scores candidates
predict_hoi.py - HOI graph network → Ranked HOI results: refines ranking
README.md - QPIC detector → Feature backbone: uses features
models/detr.py - QPIC detector → Transformer: uses transformer
models/detr.py - QPIC detector → HOI head: uses HOI head
models/detr.py - QPIC detector → Segmentation head: uses masks
models/detr.py - QPIC detector → Box operations: uses box ops
models/detr.py - HICO dataset → Image transforms: applies transforms
datasets/hico.py - COCO dataset → Image transforms: applies transforms
datasets/coco.py - V-COCO dataset → Image transforms: applies transforms
datasets/vcoco.py - QPIC training entry → Training engine: trains model
main.py - QPIC training entry → HICO dataset: loads dataset
main.py - QPIC training entry → QPIC detector: builds model
main.py - Training engine → Dataset evaluators: evaluates outputs
engine.py - Training engine → Runtime utilities: logs metrics
engine.py - GNN training → HOI graph network: trains network
train_gnn.py - GNN evaluation → HOI graph network: evaluates network
- HICO dataset → Dataset evaluators: supports evaluation
- V-COCO dataset → Dataset evaluators: supports evaluation