How riccardoriccio/fitness-ai-trainer-with-automatic-exercise-recognition-and-counting works
Project Type and Purpose
• From the README and file tree, this is a Python-based web application built with Streamlit. It serves as a “Personal AI Trainer” that:
• Provides a UI (via Streamlit) for users to upload exercise videos or use a webcam
• Performs real-time pose estimation (MediaPipe)
• Extracts features (angles/distances) and classifies exercises (LSTM/BiLSTM models)
• Counts repetitions
• Offers a chatbot interface (OpenAI API via LangChain)
File Structure
• Top-level scripts:
• main.py: entry point running the Streamlit app (UI + controller)
• ExerciseAiTrainer.py, AiTrainer_utils.py, PoseModule2.py: core CV and pose-processing modules
• chatbot.py: wraps OpenAI GPT-3.5-turbo for fitness Q&A
• extract_features.py, create_sequence_of_features.py, train_bidirectionallstm.py: offline feature extraction & model-training pipeline
• Pretrained assets:
• .h5 (model), .pkl (scaler, encoder)
• Config/environments:
• environment.yml, requirements.txt, packages.txt
Key Architecture Insights from the README
• Modular but monolithic Python backend
• Streamlit serves both frontend and backend in a single process
• MediaPipe for pose landmarks → utilities to compute angles/distances → BiLSTM classification → counting logic
• Chatbot integrated into same app, calling OpenAI API
• Offline training pipeline separated from runtime app
a. Main Components
• Streamlit UI (web client)
• Backend Orchestrator (main.py)
• Pose Estimation Module (PoseModule2)
• Feature Extraction & Preprocessing (AiTrainer_utils, extract_features)
• Classification Model (BiLSTM loaded from .h5 + .pkl)
• Repetition Counting Logic (ExerciseAiTrainer)
• Chatbot Service (chatbot.py → OpenAI API)
• Offline Training Pipeline (extract_features.py → create_sequence_of_features.py → train_bidirectionallstm.py)
b. Relationships & Data Flows
• User action (video upload / webcam) → Streamlit UI
• UI → calls backend orchestrator
• Orchestrator → PoseModule2 to get landmarks
• Landmarks → AiTrainer_utils for feature computation
• Features → classification model → predicted exercise
• Prediction + feature stream → ExerciseAiTrainer counter → repetition count
• UI displays count, form feedback, charts
• Chatbot widget → backend calls chatbot.py → external OpenAI API → response → UI
• Models & scalers loaded once at startup (shared resources)
c. Architectural Patterns / Principles
• Modular layering: UI layer, processing layer (CV & feature), ML inference layer, external API integration
• Single-process monolith (Streamlit)
• Offline/online separation: training pipeline vs inference pipeline
d. Key Technologies & Frameworks
• Streamlit (UI + server)
• MediaPipe (pose estimation)
• OpenCV (video handling)
• TensorFlow / Keras (LSTM & BiLSTM)
• LangChain + OpenAI API (Chatbot)
• Python 3.7+, virtualenv