# cseeyangchen/neuron architecture

> Architecture diagram of the GitHub repository cseeyangchen/neuron, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cseeyangchen/neuron
- Repository: https://github.com/cseeyangchen/neuron
- Video tour (about a minute): https://gitdiagram.com/cseeyangchen/neuron/video

## Overview

Neuron trains and evaluates a zero-shot skeleton action recognizer: dataset-specific skeleton preparation feeds a seen-class Shift-GCN pretraining stage and then a matching model that learns spatial and temporal prototypes for seen/unseen-class recognition. The README documents both NTU and PKU preparation and training paths; sampled code confirms the NTU feeder, Shift-GCN model, matching model, and semantic-description loading. Matching internals and some dataset stages are not fully sampled, so uncertain wiring is omitted.

Main entry points: [`main_match.py`](https://github.com/cseeyangchen/neuron/blob/main/main_match.py) (Matching training and evaluation), [`Pretrain_Shift_GCN/main.py`](https://github.com/cseeyangchen/neuron/blob/main/Pretrain_Shift_GCN/main.py) (Shift-GCN pretraining), [`model/shiftgcn_match.py`](https://github.com/cseeyangchen/neuron/blob/main/model/shiftgcn_match.py) (Spatial-temporal matching), [`data/ntu60/get_raw_skes_data.py`](https://github.com/cseeyangchen/neuron/blob/main/data/ntu60/get_raw_skes_data.py) (NTU skeleton extraction)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_prep["Skeleton preparation"]
  node_ntu_raw["NTU skeleton extraction"]
  node_ntu_clean["NTU denoising"]
  node_ntu_transform["NTU sequence transform"]
  node_pku_extract["PKU skeleton extraction"]
  node_pku_prepare["PKU data generation"]
  node_pku_downsample["PKU downsample<br/>[preprocess_pku.py]"]
  node_pku_concat["PKU split concatenation<br/>[pku_concat.py]"]
  node_prepared_data[("Prepared skeleton data")]
end

subgraph group_pretrain["Seen-class pretraining"]
  node_pretrain_entry["Shift-GCN pretraining<br/>[main.py]"]
  node_pretrain_feeder["Training data feeder<br/>[feeder_ntu.py]"]
  node_pretrain_model["Shift-GCN encoder<br/>[shift_gcn.py]"]
  node_pretrained_weights[("Encoder weights")]
end

subgraph group_recognition["Zero-shot recognition"]
  node_recognition_entry["Matching training and evaluation<br/>[main_match.py]"]
  node_recognition_feeder["Zero-shot data feeder<br/>[feeder_ntu.py]"]
  node_matching_model["Spatial-temporal matching<br/>[shiftgcn_match.py]"]
  node_semantic_descriptions[("Phase descriptions")]
  node_predictions["Action scores"]
end

subgraph group_runtime["Model internals"]
  node_skeleton_graph["Skeleton graph<br/>[ntu_rgb_d.py]"]
end

node_researcher(("Researcher"))
node_skeleton_sources[("Skeleton datasets")]

node_skeleton_sources -->|"provides skeletons"| node_ntu_raw
node_ntu_raw -->|"extracts sequences"| node_ntu_clean
node_ntu_clean -->|"filters skeletons"| node_ntu_transform
node_ntu_transform -->|"writes sequences"| node_prepared_data
node_skeleton_sources -->|"provides skeletons"| node_pku_extract
node_pku_extract -->|"generates skeletons"| node_pku_prepare
node_pku_prepare -->|"resamples frames"| node_pku_downsample
node_pku_downsample -->|"passes split data"| node_pku_concat
node_pku_concat -->|"writes combined data"| node_prepared_data
node_researcher -->|"starts pretraining"| node_pretrain_entry
node_pretrain_entry -->|"loads samples"| node_pretrain_feeder
node_pretrain_entry -->|"trains encoder"| node_pretrain_model
node_pretrain_feeder -->|"reads sequences"| node_prepared_data
node_pretrain_model -->|"uses graph layout"| node_skeleton_graph
node_pretrain_entry -->|"saves weights"| node_pretrained_weights
node_researcher -->|"starts recognition"| node_recognition_entry
node_recognition_entry -->|"loads samples"| node_recognition_feeder
node_recognition_entry -->|"trains and evaluates"| node_matching_model
node_recognition_feeder -->|"reads split data"| node_prepared_data
node_recognition_entry -->|"loads descriptions"| node_semantic_descriptions
node_matching_model -->|"initializes encoder"| node_pretrained_weights
node_matching_model -->|"uses graph layout"| node_skeleton_graph
node_recognition_entry -->|"produces scores"| node_predictions

click node_ntu_raw "https://github.com/cseeyangchen/neuron/blob/main/data/ntu60/get_raw_skes_data.py"
click node_ntu_clean "https://github.com/cseeyangchen/neuron/blob/main/data/ntu60/get_raw_denoised_data.py"
click node_ntu_transform "https://github.com/cseeyangchen/neuron/blob/main/data/ntu60/seq_transformation.py"
click node_pku_extract "https://github.com/cseeyangchen/neuron/blob/main/data/pkummd/part1/pku_part1_skeleton.py"
click node_pku_prepare "https://github.com/cseeyangchen/neuron/blob/main/data/pkummd/part1/pku_part1_gendata.py"
click node_pku_downsample "https://github.com/cseeyangchen/neuron/blob/main/data/pkummd/part1/preprocess_pku.py"
click node_pku_concat "https://github.com/cseeyangchen/neuron/blob/main/data/pkummd/part1/pku_concat.py"
click node_pretrain_entry "https://github.com/cseeyangchen/neuron/blob/main/Pretrain_Shift_GCN/main.py"
click node_pretrain_feeder "https://github.com/cseeyangchen/neuron/blob/main/Pretrain_Shift_GCN/feeders/feeder_ntu.py"
click node_pretrain_model "https://github.com/cseeyangchen/neuron/blob/main/Pretrain_Shift_GCN/model/shift_gcn.py"
click node_recognition_entry "https://github.com/cseeyangchen/neuron/blob/main/main_match.py"
click node_recognition_feeder "https://github.com/cseeyangchen/neuron/blob/main/feeders/feeder_ntu.py"
click node_matching_model "https://github.com/cseeyangchen/neuron/blob/main/model/shiftgcn_match.py"
click node_skeleton_graph "https://github.com/cseeyangchen/neuron/blob/main/graph/ntu_rgb_d.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ntu_raw,node_ntu_clean,node_ntu_transform,node_pku_extract,node_pku_prepare,node_pku_downsample,node_pku_concat,node_prepared_data toneBlue
class node_pretrain_entry,node_pretrain_feeder,node_pretrain_model,node_pretrained_weights,node_skeleton_sources toneAmber
class node_recognition_entry,node_recognition_feeder,node_matching_model,node_semantic_descriptions,node_predictions toneMint
class node_skeleton_graph toneRose
class node_researcher toneIndigo
```

## Components

### Skeleton preparation

- [NTU skeleton extraction](https://github.com/cseeyangchen/neuron/blob/main/data/ntu60/get_raw_skes_data.py): `data/ntu60/get_raw_skes_data.py`
- [NTU denoising](https://github.com/cseeyangchen/neuron/blob/main/data/ntu60/get_raw_denoised_data.py): `data/ntu60/get_raw_denoised_data.py`
- [NTU sequence transform](https://github.com/cseeyangchen/neuron/blob/main/data/ntu60/seq_transformation.py): `data/ntu60/seq_transformation.py`
- [PKU skeleton extraction](https://github.com/cseeyangchen/neuron/blob/main/data/pkummd/part1/pku_part1_skeleton.py): `data/pkummd/part1/pku_part1_skeleton.py`
- [PKU data generation](https://github.com/cseeyangchen/neuron/blob/main/data/pkummd/part1/pku_part1_gendata.py): `data/pkummd/part1/pku_part1_gendata.py`
- [PKU downsample](https://github.com/cseeyangchen/neuron/blob/main/data/pkummd/part1/preprocess_pku.py): `data/pkummd/part1/preprocess_pku.py`
- [PKU split concatenation](https://github.com/cseeyangchen/neuron/blob/main/data/pkummd/part1/pku_concat.py): `data/pkummd/part1/pku_concat.py`
- **Prepared skeleton data**

### Seen-class pretraining

- [Shift-GCN pretraining](https://github.com/cseeyangchen/neuron/blob/main/Pretrain_Shift_GCN/main.py): `Pretrain_Shift_GCN/main.py`
- [Training data feeder](https://github.com/cseeyangchen/neuron/blob/main/Pretrain_Shift_GCN/feeders/feeder_ntu.py): `Pretrain_Shift_GCN/feeders/feeder_ntu.py`
- [Shift-GCN encoder](https://github.com/cseeyangchen/neuron/blob/main/Pretrain_Shift_GCN/model/shift_gcn.py): `Pretrain_Shift_GCN/model/shift_gcn.py`
- **Encoder weights**

### Zero-shot recognition

- [Matching training and evaluation](https://github.com/cseeyangchen/neuron/blob/main/main_match.py): `main_match.py`
- [Zero-shot data feeder](https://github.com/cseeyangchen/neuron/blob/main/feeders/feeder_ntu.py): `feeders/feeder_ntu.py`
- [Spatial-temporal matching](https://github.com/cseeyangchen/neuron/blob/main/model/shiftgcn_match.py): `model/shiftgcn_match.py`
- **Phase descriptions**
- **Action scores**

### Model internals

- [Skeleton graph](https://github.com/cseeyangchen/neuron/blob/main/graph/ntu_rgb_d.py): `graph/ntu_rgb_d.py`

### Other components

- **Researcher**
- **Skeleton datasets**

## Connections

- Skeleton datasets → NTU skeleton extraction: provides skeletons
- NTU skeleton extraction → NTU denoising: extracts sequences
- NTU denoising → NTU sequence transform: filters skeletons
- NTU sequence transform → Prepared skeleton data: writes sequences
- Skeleton datasets → PKU skeleton extraction: provides skeletons
- PKU skeleton extraction → PKU data generation: generates skeletons
- PKU data generation → PKU downsample: resamples frames
- PKU downsample → PKU split concatenation: passes split data
- PKU split concatenation → Prepared skeleton data: writes combined data
- Researcher → Shift-GCN pretraining: starts pretraining
- Shift-GCN pretraining → Training data feeder: loads samples
- Shift-GCN pretraining → Shift-GCN encoder: trains encoder
- Training data feeder → Prepared skeleton data: reads sequences
- Shift-GCN encoder → Skeleton graph: uses graph layout
- Shift-GCN pretraining → Encoder weights: saves weights
- Researcher → Matching training and evaluation: starts recognition
- Matching training and evaluation → Zero-shot data feeder: loads samples
- Matching training and evaluation → Spatial-temporal matching: trains and evaluates
- Zero-shot data feeder → Prepared skeleton data: reads split data
- Matching training and evaluation → Phase descriptions: loads descriptions
- Spatial-temporal matching → Encoder weights: initializes encoder
- Spatial-temporal matching → Skeleton graph: uses graph layout
- Matching training and evaluation → Action scores: produces scores