How vipl-slp/pointlstm-gesture-recognition-pytorch works
Project type and purpose
• This is a research-oriented machine-learning library for point-cloud-based gesture recognition.
• It implements the “PointLSTM” network from CVPR 2020, providing data preprocessing, model definitions, training/inference code, and unit tests.
File structure
• Top-level directories:
• dataset/ – scripts for converting raw depth videos (NVGesture, SHREC’17) into point-cloud sequences
• experiments/ – core training/inference pipeline
– main.py orchestrates train/test loops
– models/ defines PointLSTM and related modules (motion.py, op.py, pointlstm.py)
– nvidia_dataloader.py & shrec_dataloader.py wrap processed data into PyTorch Datasets/Dataloaders
– utils/ contains submodules for FLOPs calculation, device setup, optimizer, parameter parsing, transforms, logging/recording, metrics/statistics
– pointlstm.yaml holds hyperparameters/config
• test/ – unit tests for model invariance and utility functions
• Root:
• __init__.py, LICENSE, readme.md
README
• Confirms two datasets (SHREC’17, NvGesture), outlines preprocessing steps, training/inference commands, and PyTorch (>1.0) dependency.
a. Main Components
1. Data Preparation Module
– shrec17_process.py
– nvidia_dataset_split.py & nvidia_process.py
– dataset/utils.py
2. Data Loading Module
– shrec_dataloader.py
– nvidia_dataloader.py
3. Model Module
– experiments/models/pointlstm.py
– experiments/models/pointlstmcell (in code, embedded in pointlstm.py)
– experiments/models/motion.py & op.py
4. Training & Inference Orchestrator
– experiments/main.py
– pointlstm.yaml (configuration)
5. Utilities Module
– experiments/utils/{device, optimizer, parameters, pts_transform, random_state, record, statistic, FLOPs, tools}
6. Test Suite
– test/pointlstm_invariant_test.py
– test/pointlstmcell_invariant_test.py
– test/test_utils.py
7. External Dependencies
– PyTorch, Python standard libs
b. Relationships & Interactions
1. Raw depth videos → Data Preparation Module → Processed point-cloud files
2. Processed files → Data Loading Module → PyTorch Dataset/Dataloader
3. Dataloader → Model Module → forward/backward passes
4. Model parameters + loss → Utilities (optimizer, device, random state) → weight updates
5. Metrics & logs → Utilities (record, statistic) → output in work-dir
6. Inference follows the same flow minus weight updates
7. Test Suite imports Data Loader, Model, and Utils to validate invariants
c. Architectural Patterns & Principles
• Modular separation of concerns: preprocessing, data loading, model, training, utilities, testing
• Configuration-driven (pointlstm.yaml)
• Reusable utility functions (e.g., device abstraction, optimizer factory, logging)
d. Key Technologies/Frameworks
– Python 3.x
– PyTorch (>1.0)
– YAML for config
– pytest/unittest for tests