How wjchaogit/group-activity-recognition works
Project Type & Purpose
• This is a research-oriented, full-stack machine-learning pipeline for video-based group activity recognition.
• Two datasets (Volleyball, Collective) are supported.
• There is a two-stage training process: Stage 1 (fine-tune backbone CNN), Stage 2 (fix backbone, train Graph Convolutional Network).
File Structure
• Top-level ML modules:
• backbone.py: CNN backbone feature extractor
• gcn_model.py: Graph Convolutional Network for actor relations
• base_model.py: Abstract parent for both dataset-specific models
• collective.py, volleyball.py: Dataset-specific model wrappers/combiners
• dataset.py: DataLoader for both datasets
• config.py: Hyperparameters and CLI argument definitions
• utils.py: IO, logging, metric computations
• train_net.py: Common trainer orchestrator
• Data folders:
• data/volleyball: raw images, preprocessed tracks
• data/collective: MATLAB scripts + .dat tracks
• scripts/: Stage1 and Stage2 entry points for both datasets.
• result/: Checkpoint/results storage.
• Requirements: PyTorch (0.4.1), numpy, scikit-image, RoIAlign.
Extract README Insights
• Two-stage training: Stage1 calls scripts/train_*_stage1.py → trains CNN only.
• Stage2 calls scripts/train_*_stage2.py → freezes CNN, trains GCN.
• config.py holds all tunable parameters.
Main System Components & Their Interactions
A. Data Ingestion
• Raw data (videos/frames & track annotations)
→ yields per-frame actor crops + positional info.
B. Feature Extraction
• backbone.py (ResNet or similar)
→ processes actor crops via RoIAlign
→ outputs feature vectors.
C. Relation Graph Module
• gcn_model.py
→ takes feature vectors + positional adjacency
→ builds actor relation graph
→ applies GCN layers to produce context-aware actor embeddings.
D. Classification Head
• base_model.py / dataset-specific models
→ aggregate actor embeddings
→ predict group activity labels.
E. Training Orchestrator
• train_net.py & scripts/train_*
→ parse config.py
→ pipeline: DataLoader → FeatureExtractor → (GCN) → Loss → Optimizer
→ checkpoint to result/.
F. Utilities & Configuration
• config.py: All hyperparams, paths, training flags
• utils.py: Logging, checkpoint save/load, metric reporting.
Architectural Patterns & Principles
• Modularity: separate data, feature extractor, GCN, head, trainer.
• Two-stage training: pipeline can toggle GCN on/off.
• Extensibility: add new datasets by writing a new dataset loader & model wrapper.
• Configuration-driven: use config.py to control architecture, learning rate, dataset paths.