How avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises works
Project Type and Purpose
• Type: Open‐source research framework/pipeline for action quality assessment.
• Purpose: Ingest skeletal joint data (Vicon/Kinect), pre‐process and optionally reduce dimensionality, compute distance‐based performance metrics, train deep learning models (Spatio‐Temporal NN, CNN, RNN, Autoencoder), and output quality scores of rehabilitation exercises.
File Structure
• Top‐level folders:
• Data: raw and precomputed CSV inputs/outputs.
• Distance Functions: MATLAB scripts implementing metrics with/without dimensionality reduction.
• Neural Networks: Jupyter notebooks and Python loaders for Keras models (SpatioTemporalNN, CNN, RNN, Autoencoder).
• Utility Functions: MATLAB helper functions (GMM, EM, PDF, etc.).
• Results, License, README, auxiliary docs.
• Patterns:
• A linear pipeline: Data → Preprocessing → (Optional) Dimensionality Reduction → Distance Metrics → Neural Models → Evaluation.
• Dual‐language ecosystem: MATLAB for data prep & metrics; Python/Keras for deep models.
• Config scripts: Prepare_Data_for_NN.m, Prepare_Labels_for_NN.m, Autoencoder_Dims_Reduction.ipynb.
README for Additional Insights
• Confirms the use of UI‐PRMD dataset (Vicon skeletons).
• Lists the sequence of steps: prepare data, reduce dims, prepare labels, train/predict models.
• Mentions alternative skip‐steps if precomputed files are used.
• Highlights five model variants: SpatioTemporalNN_Vicon, SpatioTemporalNN_Kinect, CNN_Vicon, RNN_Vicon, Autoencoder_Dims_Reduction.
1) “Raw Data Sources”
– Vicon (.csv) and Kinect (.csv).
2) “Data Preprocessing (MATLAB)”
– Prepare_Data_for_NN.m → alignment, centering.
– Prepare_Labels_for_NN.m → scoring function.
3) “Dimensionality Reduction”
– MATLAB Autoencoder (Autoencoder_Output_*.csv).
– Python Autoencoder_Dims_Reduction.ipynb.
– PCA, Maximum Variance, No Reduction variants.
4) “Distance Function Module (MATLAB)”
– Autoencoder, PCA, MaxVar, NoDim subfolders.
– DTW, Euclidean, Mahalanobis, Log‐likelihood metrics.
5) “Neural Network Models (Python/Keras)”
– SpatioTemporalNN_Vicon.ipynb
– SpatioTemporalNN_Kinect.ipynb
– CNN_Vicon.ipynb
– RNN_Vicon.ipynb
6) “Evaluation & Results”
– Results folder, predicted quality scores.
• Raw Data → Data Preprocessing → (optionally) Dimensionality Reduction → Distance Functions → Neural Network Models → Output Scores → Results.
• Preprocessed Data also feeds directly into Neural Network Models (for no‐reduction experiments).
• Dimensionality Reduction outputs feed both distance functions (for metric‐based scoring experiments) and as inputs to deep models (autoencoder pretraining).
c. Architectural Patterns & Principles:
• Modular pipeline: each stage is language‐specific but loosely coupled via CSV artifacts.
• Experiment variants: switchable modules for reduction and metrics.
• Reproducibility: precomputed data outputs allow skipping heavy steps.
• Separation of concerns: data prep vs metric computation vs model training.
d. Technologies & Frameworks:
• MATLAB R2018+ for data prep and distance metrics (m‐files).
• Python 3 with Keras/TensorFlow for neural models.
• Jupyter Notebooks for model prototyping.
• CSV for data interchange.
• UI‐PRMD dataset (external).
Detail
• For higher‐level overviews, collapse subfolders into single grouped components (e.g., “Distance Functions”).
• Data formats and dimensions (e.g., “117‐dim skeleton → 10‐dim Autoencoder output”).
• Evaluation metrics (MSE, correlation) near the Results node.