How suferqin/fast-qpu works
Project Type and Purpose
• This project is an experimental research framework focused on learning disentangled representations in SO(3) using fast quaternion product units (QPUs). It implements experiments for both synthetic data (found in the CubeEdge folder) and real-world data. In addition, it includes extensive point cloud experiments by modifying established projects (PointNet++ and RSCNN).
• The README makes clear that the framework supports multiple data modalities: synthetic experiments with CubeEdge, real-world action recognition with datasets like FPHA and NTU RGB+D, and point cloud processing. The custom CUDA modules (in _ext_qpu) further emphasize high-performance calculations.
Analysis of the File Structure
• At the top level, there are directories for configuration (configs), data handling (data), experiments and models (CubeEdge, models, utils), and CUDA extensions (_ext_qpu).
• The project also integrates two separate point cloud experiment pipelines grouped under the “pointcloud” directory which is subdivided into the PointNet++ and RSCNN subprojects.
• The presence of typical training scripts (train.py, test.py) and configuration files (YAML files in configs) indicates a clear separation of experiment configuration, model implementation, training, and testing.
• Utility directories (like utils, data/scripts) point to data preprocessing, visualization, and logging capabilities.
• The _ext_qpu folder contains custom C++/CUDA implementations – generally built via setup.py – which are crucial for the performance of the quaternion units.
Key Components and Their Relationships
a. Core Experiment Modules:
• CubeEdge: Contains the main experimental notebook (experiments.ipynb), the primary source code (cube_edge.py), and training utilities (train_utils.py).
• Models: The models folder holds implementations of various architectures (e.g., agc_lstm, dgnn, mlp_lstm, etc.) designed for both synthetic and real-world experiments.
• Configs: YAML configuration files dictate experiment parameters; these are read by train.py, test.py, and other control scripts.
• Data: Includes dataset-specific modules (feeder scripts, info files) and scripts for data generation, visualizations, statistics, and edge generation.
• Utilities: Common helper functions (logger, quaternion operations, skeleton utilities) that abstract common logic used across experiments.
b. Custom High-Performance Layers:
• _ext_qpu: This directory includes the C++/CUDA source and header files (bindings, hmd, cuda_utils) – these represent the performance-critical QPU implementations, compiled as extensions.
• Integration: These lower-level modules are likely wrapped and invoked from the high-level Python model implementations (e.g., in utils or models).
c. Point Cloud Experiment Modules:
• PointNet++: Contains its own set of training scripts, model definitions, data loaders, and CUDA-extension source code for point cloud segmentation and classification.
• RSCNN: Similar structure to PointNet++ with specialized setup for point cloud data, including different configuration files, training scripts, and custom operations.
• These subprojects are essentially self-contained but are integrated within the project umbrella for experiments on QPU layers in different modalities.