How kaai520/pgfa works
PGFA trains skeleton encoders to align action features with text embeddings, then evaluates unseen actions using prototype-guided alignment. It supports ST-GCN and Shift-GCN zero-shot settings, plus ST-GCN one-shot experiments. The README documents prepared skeleton data, label/description text features, and prototype-based testing; sampled source confirms the training components and backbone implementations. Test and one-shot wiring is represented at the workflow level because those implementations were not fully sampled.
Main entry points: main.py Zero-shot training · dataset.py Skeleton dataset · test_main.py Zero-shot testing · one_shot_test_main.py One-shot testing
Components 19
Experiment workflows
- Zero-shot training
main.py - Zero-shot testing
test_main.py - One-shot workflow
one_shot_main.py - One-shot testing
one_shot_test_main.py - Experiment settings
config.py
Data and language
- Skeleton dataset
dataset.py - Prepared data
- Action descriptions
descriptions - Text embeddings
- Sentence embedding
sentence_bert_embedding.py
Feature models
- ST-GCN
module/gcn/st_gcn.py - Shift-GCN
module/shift_gcn.py - Feature adapter
module/adapter.py
Alignment and evaluation
- Contrastive losses
KLLoss.py - Similarity and metrics
tool.py - Prototype alignment
test_main.py - Predictions and scores
Other components
- Researcher
- Sentence-BERT
Connections 24
- Researcher → Zero-shot training: starts training
- Researcher → Zero-shot testing: starts testing
- Researcher → One-shot workflow: starts training
- Experiment settings → Zero-shot training: configures
- Experiment settings → Zero-shot testing: configures
- Skeleton dataset → Prepared data: loads skeletons
- Skeleton dataset → Text embeddings: loads features
- Action descriptions → Sentence embedding: supplies text
- Sentence embedding → Sentence-BERT: encodes descriptions
- Sentence embedding → Text embeddings: writes vectors
- Zero-shot training → Skeleton dataset: loads batches
- Zero-shot training → ST-GCN: trains encoder
- Zero-shot training → Shift-GCN: selects encoder
- Zero-shot training → Feature adapter: trains adapter
- Zero-shot training → Contrastive losses: computes loss
- Zero-shot testing → Skeleton dataset: loads evaluation data
- Zero-shot testing → Prototype alignment: evaluates alignment
- Prototype alignment → Similarity and metrics: scores features
- Prototype alignment → Predictions and scores: produces scores
- One-shot workflow → Skeleton dataset: loads support data
- One-shot testing → Skeleton dataset: loads test data
- One-shot testing → Prototype alignment: uses exemplars
- One-shot testing → Similarity and metrics: scores predictions
- Zero-shot training → Predictions and scores: saves checkpoints