How yueliao/cdn works
Project Type and Purpose
• This is a research-oriented deep learning codebase for Human-Object Interaction (HOI) detection.
• It implements a hybrid “two-stage + one-stage” detection network (CDN), built on top of DETR, PPDM and QPIC ideas, for HICO-DET and V-COCO benchmarks.
File Structure
• Top-level scripts:
• main.py: entry point for training/evaluation
• engine.py: training & evaluation loops, distributed setup
• convert_parameters.py: prepares DETR pretrained weights
• convert_vcoco_annotations.py & generate_vcoco_official.py: data conversion & metric generation
• Directories:
• datasets/: data loaders & transforms for HICO and V-COCO
• models/: building blocks — backbone, position encoding, matcher, CDN HOI heads
• paper_images/: contains the published framework illustration
• Config & metadata:
• requirements.txt, README.md, LICENSE
Key Insights from README
• The framework image outlines data prep → model (backbone + two decoders) → loss/matcher → output.
• Supports multi-GPU training via torch.distributed.launch.
• Two-step training: base model → dynamic re-weighting fine-tune.
• Evaluation includes both built-in scripts and an external V-COCO eval module.
Main Components
a. Data Preparation & Conversion
• External data sources (HICO-DET, V-COCO)
• Conversion scripts (convert_vcoco_annotations.py, convert_parameters.py)
• Local data store (data/hico_20160224_det, data/v-coco)
b. Data Pipeline
• datasets/*.py: Dataset classes + transforms
• DataLoader → batching
c. Model
• Backbone (ResNet50)
• Encoding
• Query-based DETR decoder for object proposals
• Two HOI decoders:
• hopd (two-stage)
• interaction (one-stage)
• Matcher (Hungarian algorithm)
d. Training Engine
• engine.py: forward, compute losses, backward, optimizer step
• Distributed training orchestration
e. Evaluation & Output
• Built-in eval via main.py with --eval
• Official V-COCO eval (generate_vcoco_official.py + vsrl_eval.py)
• Outputs: logs/, checkpoint files, .pickle results
Relationships & Data Flow
• From local data → Dataset loader → model input.
• Within model: image → backbone → feature maps → queries → HOI decoders → predictions.
• Predictions → matcher → loss → engine → optimized weights → checkpoints.
• Checkpoints → evaluation scripts → metrics.
Architectural Patterns & Design Principles
• Modular separation: dataset code vs. model vs. engine vs. utilities.
• Dependency injection: passing configs/paths via CLI arguments.
• Reuse of external libraries: DETR, PPDM, PyTorch Distributed.
• Two-stage pipeline (proposal + interaction) combined with one-stage pipeline (end-to-end queries).
Technologies & Frameworks
• Python 3 (with Python 2 requirement only for V-COCO conversion)
• PyTorch + torchvision
• Torch distributed for multi-GPU
• External repos: DETR, PPDM, QPIC