How chrisprasanna/exercise_recognition_ai works
Main Components
• Webcam/Video Input Module:
• Represents the source of real-time video feeds.
• Utilizes OpenCV to capture frames from the user’s webcam.
• Pose Estimation Module:
• Uses MediaPipe’s BlazePose (or a similar pretrained CNN model) to extract pose landmarks from the video frames.
• Data Preprocessing & Feature Extraction:
• Processes the pose coordinates by computing joint angles (from relative landmark positions).
• Includes a guided data collection pipeline to generate training datasets.
• Deep Learning Models:
• Contains the custom LSTM model and the Attention-based LSTM model (with pre-trained weights in the “models” directory).
• These components interpret the processed pose data and output exercise recognition predictions.
• Visualization & Rep Counting Module:
• Displays live rep counts, joint angles, and prediction probability distributions.
• Implements heuristics for tracking exercise performance.
• Web Application Interface:
• Built using Streamlit which wraps around the above methods.
• Provides a user-friendly GUI to interact with the AI.
• Deployment & CI/CD Pipeline:
• Dockerfile and .dockerignore manage containerization.
• GitHub workflows (CI.yml under .github/workflows) automate the building and deployment process.
• Integration with Heroku (and optionally, Streamlit Cloud) for hosting the web app.
• Supporting Modules:
• The “tests” directory (e.g., feature_engineering.ipynb) for validating feature extraction and performance.
• Research documents (PDFs in the “research” folder) as supplemental information on the methodology.
How Components Interact
• Video Input → Pose Estimation → Data Preprocessing → Deep Learning Models
• Then branch out to two outputs:
• Visualization Component that displays the analysis (joint angles, rep count, prediction probabilities).
• Web Interface that integrates all the above components and presents them to the user.
• Optionally, include a side branch for the guided data collection pipeline feeding data back to model training.
• Additionally, include a separate section showing the CI/CD pipeline:
• How changes trigger GitHub Actions to build a Docker image, deploy to Heroku, and maintain version control.
Architectural Patterns and Design Principles
• Separation of Concerns:
• Clearly separate the data acquisition/processing modules from the model inference and user interface.
• Modularity:
• Treat the pose estimation, deep learning prediction, and visualization components as discrete modules which can be updated independently.
• Deployment Pipeline:
• How containerization (with Docker) and automation (with GitHub Actions) fit into the overall software lifecycle.
• Technology Stack:
• Frameworks/libraries such as OpenCV, MediaPipe, TensorFlow/Keras, Streamlit, Docker, and CI/CD tools (GitHub Actions).