# vipl-slp/pointlstm-gesture-recognition-pytorch architecture

> Architecture diagram of the GitHub repository vipl-slp/pointlstm-gesture-recognition-pytorch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vipl-slp/pointlstm-gesture-recognition-pytorch
- Repository: https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch

## Overview

##### Project type and purpose
- This is a research-oriented machine-learning library for point-cloud-based gesture recognition.
- It implements the “PointLSTM” network from CVPR 2020, providing data preprocessing, model definitions, training/inference code, and unit tests.

##### File structure
- Top-level directories:
- dataset/ – scripts for converting raw depth videos (NVGesture, SHREC’17) into point-cloud sequences
- experiments/ – core training/inference pipeline
– main.py orchestrates train/test loops
– models/ defines PointLSTM and related modules (motion.py, op.py, pointlstm.py)
– nvidia_dataloader.py & shrec_dataloader.py wrap processed data into PyTorch Datasets/Dataloaders
– utils/ contains submodules for FLOPs calculation, device setup, optimizer, parameter parsing, transforms, logging/recording, metrics/statistics
– pointlstm.yaml holds hyperparameters/config
- test/ – unit tests for model invariance and utility functions
- Root:
- __init__.py, LICENSE, readme.md

##### README
- Confirms two datasets (SHREC’17, NvGesture), outlines preprocessing steps, training/inference commands, and PyTorch (>1.0) dependency.

a. Main Components
1. Data Preparation Module
– shrec17_process.py
– nvidia_dataset_split.py & nvidia_process.py
– dataset/utils.py
2. Data Loading Module
– shrec_dataloader.py
– nvidia_dataloader.py
3. Model Module
– experiments/models/pointlstm.py
– experiments/models/pointlstmcell (in code, embedded in pointlstm.py)
– experiments/models/motion.py & op.py
4. Training & Inference Orchestrator
– experiments/main.py
– pointlstm.yaml (configuration)
5. Utilities Module
– experiments/utils/{device, optimizer, parameters, pts_transform, random_state, record, statistic, FLOPs, tools}
6. Test Suite
– test/pointlstm_invariant_test.py
– test/pointlstmcell_invariant_test.py
– test/test_utils.py
7. External Dependencies
– PyTorch, Python standard libs

b. Relationships & Interactions
1. Raw depth videos → Data Preparation Module → Processed point-cloud files
2. Processed files → Data Loading Module → PyTorch Dataset/Dataloader
3. Dataloader → Model Module → forward/backward passes
4. Model parameters + loss → Utilities (optimizer, device, random state) → weight updates
5. Metrics & logs → Utilities (record, statistic) → output in work-dir
6. Inference follows the same flow minus weight updates
7. Test Suite imports Data Loader, Model, and Utils to validate invariants

c. Architectural Patterns & Principles
- Modular separation of concerns: preprocessing, data loading, model, training, utilities, testing
- Configuration-driven (pointlstm.yaml)
- Reusable utility functions (e.g., device abstraction, optimizer factory, logging)

d. Key Technologies/Frameworks
– Python 3.x
– PyTorch (>1.0)
– YAML for config
– pytest/unittest for tests

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Dependencies
    ExtPyTorch["PyTorch (>1.0)"]:::external
    ExtPython["Python 3.x"]:::external

    %% Raw Data Source
    RawVideos["Raw Depth Videos"]:::data

    %% Data Preparation Module
    subgraph "Data Preparation Module" 
        direction TB
        DP1["nvidia_dataset_split.py"]:::data
        DP2["nvidia_process.py"]:::data
        DP3["shrec17_process.py"]:::data
        DP4["utils.py"]:::data
    end

    %% Processed Data
    ProcessedData["Processed Point-Cloud Files"]:::data

    %% Data Loading Module
    subgraph "Data Loading Module"
        direction TB
        DL1["nvidia_dataloader.py"]:::data
        DL2["shrec_dataloader.py"]:::data
    end

    %% Configuration
    Config["pointlstm.yaml"]:::config

    %% Model Module
    subgraph "Model Module"
        direction TB
        M1["pointlstm.py"]:::model
        M2["motion.py"]:::model
        M3["op.py"]:::model
    end

    %% Training & Inference Orchestrator
    subgraph "Training/Inference Orchestrator"
        direction TB
        Main["main.py"]:::model
        ConfigNode["Config"]:::config
    end

    %% Utilities Module
    subgraph "Utilities Module"
        direction TB
        U1["device.py"]:::utils
        U2["optimizer.py"]:::utils
        U3["parameters.py"]:::utils
        U4["pts_transform.py"]:::utils
        U5["random_state.py"]:::utils
        U6["record.py"]:::utils
        U7["statistic.py"]:::utils
        U8["FLOPs.py"]:::utils
        U9["tools.py"]:::utils
    end

    %% Logs & Metrics Output
    Outputs["Logs & Metrics"]:::utils

    %% Test Suite
    subgraph "Test Suite"
        direction TB
        T1["pointlstm_invariant_test.py"]:::test
        T2["pointlstmcell_invariant_test.py"]:::test
        T3["test_utils.py"]:::test
    end

    %% Flows
    ExtPyTorch --> RawVideos
    ExtPython --> RawVideos
    RawVideos -->|"raw depth videos"| DP1
    RawVideos -->|"raw depth videos"| DP2
    RawVideos -->|"raw depth videos"| DP3
    DP1 --> ProcessedData
    DP2 --> ProcessedData
    DP3 --> ProcessedData
    DP4 --> ProcessedData
    ProcessedData -->|"Processed Point Clouds"| DL1
    ProcessedData -->|"Processed Point Clouds"| DL2
    Config --> DL1
    Config --> DL2
    DL1 -->|"Mini-batch Tensors"| M1
    DL2 -->|"Mini-batch Tensors"| M1
    M1 -->|"forward/backward"| Main
    M2 --> M1
    M3 --> M1
    Config --> Main
    Main --> U1
    Main --> U2
    Main --> U3
    Main --> U4
    Main --> U5
    Main --> U6
    Main --> U7
    Main --> U8
    Main --> U9
    U2 --> Main
    Main --> Outputs
    U6 --> Outputs
    U7 --> Outputs

    %% Test Invocations
    T1 --> M1
    T1 --> U4
    T2 --> M1
    T3 --> U1
    T3 --> U2
    T3 --> U7

    %% Click Events
    click DP1 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/dataset/nvidia_dataset_split.py"
    click DP2 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/dataset/nvidia_process.py"
    click DP3 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/dataset/shrec17_process.py"
    click DP4 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/dataset/utils.py"
    click DL1 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/nvidia_dataloader.py"
    click DL2 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/shrec_dataloader.py"
    click M1 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/models/pointlstm.py"
    click M2 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/models/motion.py"
    click M3 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/models/op.py"
    click Main "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/main.py"
    click Config "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/pointlstm.yaml"
    click U1 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/utils/device.py"
    click U2 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/utils/optimizer.py"
    click U3 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/utils/parameters.py"
    click U4 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/utils/pts_transform.py"
    click U5 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/utils/random_state.py"
    click U6 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/utils/record.py"
    click U7 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/utils/statistic.py"
    click U8 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/utils/FLOPs.py"
    click U9 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/experiments/utils/tools.py"
    click T1 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/test/pointlstm_invariant_test.py"
    click T2 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/test/pointlstmcell_invariant_test.py"
    click T3 "https://github.com/vipl-slp/pointlstm-gesture-recognition-pytorch/blob/master/test/test_utils.py"

    %% Styles
    classDef data fill:#D0E6FF,stroke:#004080,color:#004080
    classDef model fill:#DFFFE0,stroke:#028A0F,color:#028A0F
    classDef utils fill:#FFE8CC,stroke:#E65C00,color:#E65C00
    classDef test fill:#E0E0E0,stroke:#888888,color:#555555
    classDef config fill:#FFF2CC,stroke:#DDB600,color:#B38600
    classDef external fill:#F5F5F5,stroke:#AAAAAA,color:#666666
```
