# avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises architecture

> Architecture diagram of the GitHub repository avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises
- Repository: https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises

## Overview

##### Project Type and Purpose
- Type: Open‐source research framework/pipeline for action quality assessment.
- Purpose: Ingest skeletal joint data (Vicon/Kinect), pre‐process and optionally reduce dimensionality, compute distance‐based performance metrics, train deep learning models (Spatio‐Temporal NN, CNN, RNN, Autoencoder), and output quality scores of rehabilitation exercises.

##### File Structure
- Top‐level folders:
- Data: raw and precomputed CSV inputs/outputs.
- Distance Functions: MATLAB scripts implementing metrics with/without dimensionality reduction.
- Neural Networks: Jupyter notebooks and Python loaders for Keras models (SpatioTemporalNN, CNN, RNN, Autoencoder).
- Utility Functions: MATLAB helper functions (GMM, EM, PDF, etc.).
- Results, License, README, auxiliary docs.
- Patterns:
- A linear pipeline: Data → Preprocessing → (Optional) Dimensionality Reduction → Distance Metrics → Neural Models → Evaluation.
- Dual‐language ecosystem: MATLAB for data prep & metrics; Python/Keras for deep models.
- Config scripts: Prepare_Data_for_NN.m, Prepare_Labels_for_NN.m, Autoencoder_Dims_Reduction.ipynb.

##### README for Additional Insights
- Confirms the use of UI‐PRMD dataset (Vicon skeletons).
- Lists the sequence of steps: prepare data, reduce dims, prepare labels, train/predict models.
- Mentions alternative skip‐steps if precomputed files are used.
- Highlights five model variants: SpatioTemporalNN_Vicon, SpatioTemporalNN_Kinect, CNN_Vicon, RNN_Vicon, Autoencoder_Dims_Reduction.

1) “Raw Data Sources”
– Vicon (.csv) and Kinect (.csv).
2) “Data Preprocessing (MATLAB)”
– Prepare_Data_for_NN.m → alignment, centering.
– Prepare_Labels_for_NN.m → scoring function.
3) “Dimensionality Reduction”
– MATLAB Autoencoder (Autoencoder_Output_*.csv).
– Python Autoencoder_Dims_Reduction.ipynb.
– PCA, Maximum Variance, No Reduction variants.
4) “Distance Function Module (MATLAB)”
– Autoencoder, PCA, MaxVar, NoDim subfolders.
– DTW, Euclidean, Mahalanobis, Log‐likelihood metrics.
5) “Neural Network Models (Python/Keras)”
– SpatioTemporalNN_Vicon.ipynb
– SpatioTemporalNN_Kinect.ipynb
– CNN_Vicon.ipynb
– RNN_Vicon.ipynb
6) “Evaluation & Results”
– Results folder, predicted quality scores.
- Raw Data → Data Preprocessing → (optionally) Dimensionality Reduction → Distance Functions → Neural Network Models → Output Scores → Results.
- Preprocessed Data also feeds directly into Neural Network Models (for no‐reduction experiments).
- Dimensionality Reduction outputs feed both distance functions (for metric‐based scoring experiments) and as inputs to deep models (autoencoder pretraining).
c. Architectural Patterns & Principles:
- Modular pipeline: each stage is language‐specific but loosely coupled via CSV artifacts.
- Experiment variants: switchable modules for reduction and metrics.
- Reproducibility: precomputed data outputs allow skipping heavy steps.
- Separation of concerns: data prep vs metric computation vs model training.
d. Technologies & Frameworks:
- MATLAB R2018+ for data prep and distance metrics (m‐files).
- Python 3 with Keras/TensorFlow for neural models.
- Jupyter Notebooks for model prototyping.
- CSV for data interchange.
- UI‐PRMD dataset (external).

##### Detail
- For higher‐level overviews, collapse subfolders into single grouped components (e.g., “Distance Functions”).
- Data formats and dimensions (e.g., “117‐dim skeleton → 10‐dim Autoencoder output”).
- Evaluation metrics (MSE, correlation) near the Results node.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Raw Data
    subgraph "Raw Data Sources"
        RawCorrect["Data_Correct.csv"]:::data
        RawIncorrect["Data_Incorrect.csv"]:::data
        AutoEncCorrect["Autoencoder_Output_Correct.csv"]:::data
        AutoEncIncorrect["Autoencoder_Output_Incorrect.csv"]:::data
        LabelsCorrect["Labels_Correct.csv"]:::data
        LabelsIncorrect["Labels_Incorrect.csv"]:::data
    end

    %% Data Preprocessing
    subgraph "Data Preprocessing (MATLAB)"
        PrepData["Prepare_Data_for_NN.m"]:::matlab
        PrepLabels["Prepare_Labels_for_NN.m"]:::matlab
    end

    %% Dimensionality Reduction
    subgraph "Dimensionality Reduction"
        PyAutoEnc["Autoencoder_Dims_Reduction.ipynb"]:::python
        DataLoader["M01ViconDataLoader.py"]:::python
    end

    %% Distance Functions
    subgraph "Distance Functions (MATLAB)"
        AEFolder["Autoencoder Metrics"]:::matlab
        MVFolder["Maximum Variance Metrics"]:::matlab
        NoDimFolder["No Reduction Metrics"]:::matlab
        PCAFolder["PCA Metrics"]:::matlab
    end

    %% Neural Network Models
    subgraph "Neural Network Models (Python/Keras)"
        subgraph "Spatio-Temporal NN for Vicon"
            STVIconNB["SpatioTemporalNN_Vicon.ipynb"]:::python
            DataViconLoad["DataViconLoad.py"]:::python
        end
        subgraph "Spatio-Temporal NN for Kinect v2"
            STKinectNB["SpatioTemporalNN_Kinect.ipynb"]:::python
            DataLoadKinect["Data_Load.py"]:::python
            TrainX["Train_X.csv"]:::data
            TrainY["Train_Y.csv"]:::data
        end
        subgraph "CNN & RNN for Vicon"
            CNNV["CNN_Vicon.ipynb"]:::python
            RNNV["RNN_Vicon.ipynb"]:::python
        end
    end

    %% Utility Functions
    subgraph "Utility Functions (MATLAB)"
        gaussPDF["gaussPDF.m"]:::matlab
        loglik["loglik.m"]:::matlab
        EMcov["EM_boundingCov.m"]:::matlab
        EMinit["EM_init_regularTiming.m"]:::matlab
        plotGMM1["plotGMM1.m"]:::matlab
    end

    %% Evaluation & Results
    subgraph "Evaluation & Results"
        ResultsFolder["Results/"]:::data
        README["README.md"]:::data
        License["MIT.txt"]:::data
    end

    %% Data Flow
    RawCorrect -->|raw CSV| PrepData
    RawIncorrect -->|raw CSV| PrepData
    PrepData -->|aligned CSV| PrepLabels
    PrepLabels -->|labeled CSV| PyAutoEnc
    PrepLabels -->|labeled CSV| STVIconNB
    PrepLabels -->|labeled CSV| STKinectNB
    PrepLabels -->|labeled CSV| CNNV
    PrepLabels -->|labeled CSV| RNNV

    PyAutoEnc -->|AE output CSV| AEFolder
    DataLoader -->|loaded data| STVIconNB
    AutoEncCorrect -->|precomputed| AEFolder
    AutoEncIncorrect -->|precomputed| AEFolder
    AutoEncCorrect -->|precomputed| STVIconNB
    AutoEncIncorrect -->|precomputed| STVIconNB

    AEFolder -->|distance scores| ResultsFolder
    MVFolder -->|distance scores| ResultsFolder
    NoDimFolder -->|distance scores| ResultsFolder
    PCAFolder -->|distance scores| ResultsFolder

    STVIconNB -->|predicted scores| ResultsFolder
    DataViconLoad -->|data feed| STVIconNB
    STKinectNB -->|predicted scores| ResultsFolder
    DataLoadKinect -->|data feed| STKinectNB
    CNNV -->|predicted scores| ResultsFolder
    RNNV -->|predicted scores| ResultsFolder

    %% Click Events - Raw Data
    click RawCorrect "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Data/Data_Correct.csv"
    click RawIncorrect "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Data/Data_Incorrect.csv"
    click AutoEncCorrect "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Data/Autoencoder_Output_Correct.csv"
    click AutoEncIncorrect "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Data/Autoencoder_Output_Incorrect.csv"
    click LabelsCorrect "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Data/Labels_Correct.csv"
    click LabelsIncorrect "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Data/Labels_Incorrect.csv"

    %% Click Events - Data Preprocessing
    click PrepData "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Data/Prepare_Data_for_NN.m"
    click PrepLabels "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Data/Prepare_Labels_for_NN.m"

    %% Click Events - Dimensionality Reduction
    click PyAutoEnc "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/Dimensionality Reduction for Vicon/Autoencoder_Dims_Reduction.ipynb"
    click DataLoader "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/Dimensionality Reduction for Vicon/M01ViconDataLoader.py"

    %% Click Events - Distance Functions folders
    click AEFolder "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/tree/master/Distance Functions/Autoencoder"
    click MVFolder "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/tree/master/Distance Functions/Maximum Variance"
    click NoDimFolder "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/tree/master/Distance Functions/No Dimensionality Reduction"
    click PCAFolder "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/tree/master/Distance Functions/PCA"

    %% Click Events - Neural Network Models
    click STVIconNB "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/Spatio-Temporal NN for Vicon/SpatioTemporalNN_Vicon.ipynb"
    click DataViconLoad "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/Spatio-Temporal NN for Vicon/DataViconLoad.py"
    click STKinectNB "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/Spatio-Temporal NN for Kinect v2/SpatioTemporalNN_Kinect.ipynb"
    click DataLoadKinect "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/Spatio-Temporal NN for Kinect v2/Data_Load.py"
    click TrainX "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/Spatio-Temporal NN for Kinect v2/Data_KIMORE_e5/Train_X.csv"
    click TrainY "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/Spatio-Temporal NN for Kinect v2/Data_KIMORE_e5/Train_Y.csv"
    click CNNV "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/CNN, RNN for Vicon/CNN_Vicon.ipynb"
    click RNNV "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Neural Networks/CNN, RNN for Vicon/RNN_Vicon.ipynb"

    %% Click Events - Utility Functions
    click gaussPDF "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Utility Functions/gaussPDF.m"
    click loglik "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Utility Functions/loglik.m"
    click EMcov "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Utility Functions/EM_boundingCov.m"
    click EMinit "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Utility Functions/EM_init_regularTiming.m"
    click plotGMM1 "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/Utility Functions/plotGMM1.m"

    %% Click Events - Evaluation & Results
    click ResultsFolder "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/tree/master/Results/"
    click README "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/README.md"
    click License "https://github.com/avakanski/a-deep-learning-framework-for-assessing-physical-rehabilitation-exercises/blob/master/License - MIT.txt"

    %% Styles
    classDef matlab fill:#D0E1F9,stroke:#034f84,color:#034f84;
    classDef python fill:#D6F1D6,stroke:#2E7D32,color:#2E7D32;
    classDef data fill:#E0E0E0,stroke:#888888,color:#333333;
    classDef util fill:#F9E79F,stroke:#B7950B,color:#795548;
```
