How cseeyangchen/neuron works
Neuron trains and evaluates a zero-shot skeleton action recognizer: dataset-specific skeleton preparation feeds a seen-class Shift-GCN pretraining stage and then a matching model that learns spatial and temporal prototypes for seen/unseen-class recognition. The README documents both NTU and PKU preparation and training paths; sampled code confirms the NTU feeder, Shift-GCN model, matching model, and semantic-description loading. Matching internals and some dataset stages are not fully sampled, so uncertain wiring is omitted.
Main entry points: main_match.py Matching training and evaluation · main.py Shift-GCN pretraining · shiftgcn_match.py Spatial-temporal matching · get_raw_skes_data.py NTU skeleton extraction
Components 20
Skeleton preparation
- NTU skeleton extraction
data/ntu60/get_raw_skes_data.py - NTU denoising
data/ntu60/get_raw_denoised_data.py - NTU sequence transform
data/ntu60/seq_transformation.py - PKU skeleton extraction
data/pkummd/part1/pku_part1_skeleton.py - PKU data generation
data/pkummd/part1/pku_part1_gendata.py - PKU downsample
data/pkummd/part1/preprocess_pku.py - PKU split concatenation
data/pkummd/part1/pku_concat.py - Prepared skeleton data
Seen-class pretraining
- Shift-GCN pretraining
Pretrain_Shift_GCN/main.py - Training data feeder
Pretrain_Shift_GCN/feeders/feeder_ntu.py - Shift-GCN encoder
Pretrain_Shift_GCN/model/shift_gcn.py - Encoder weights
Zero-shot recognition
- Matching training and evaluation
main_match.py - Zero-shot data feeder
feeders/feeder_ntu.py - Spatial-temporal matching
model/shiftgcn_match.py - Phase descriptions
- Action scores
Model internals
- Skeleton graph
graph/ntu_rgb_d.py
Other components
- Researcher
- Skeleton datasets
Connections 23
- Skeleton datasets → NTU skeleton extraction: provides skeletons
- NTU skeleton extraction → NTU denoising: extracts sequences
- NTU denoising → NTU sequence transform: filters skeletons
- NTU sequence transform → Prepared skeleton data: writes sequences
- Skeleton datasets → PKU skeleton extraction: provides skeletons
- PKU skeleton extraction → PKU data generation: generates skeletons
- PKU data generation → PKU downsample: resamples frames
- PKU downsample → PKU split concatenation: passes split data
- PKU split concatenation → Prepared skeleton data: writes combined data
- Researcher → Shift-GCN pretraining: starts pretraining
- Shift-GCN pretraining → Training data feeder: loads samples
- Shift-GCN pretraining → Shift-GCN encoder: trains encoder
- Training data feeder → Prepared skeleton data: reads sequences
- Shift-GCN encoder → Skeleton graph: uses graph layout
- Shift-GCN pretraining → Encoder weights: saves weights
- Researcher → Matching training and evaluation: starts recognition
- Matching training and evaluation → Zero-shot data feeder: loads samples
- Matching training and evaluation → Spatial-temporal matching: trains and evaluates
- Zero-shot data feeder → Prepared skeleton data: reads split data
- Matching training and evaluation → Phase descriptions: loads descriptions
- Spatial-temporal matching → Encoder weights: initializes encoder
- Spatial-temporal matching → Skeleton graph: uses graph layout
- Matching training and evaluation → Action scores: produces scores