How rutu-0725/fitness_assistant works
Project Overview
• The project is an AI-powered fitness coach that processes live video from an Android device to provide real-time feedback on exercise form.
• The application employs MediaPipe and OpenCV libraries for pose estimation and video processing, along with machine learning techniques using provided training (train_angle.csv) and testing (test_angle.csv and yoga_Dataset/angle.csv) datasets.
• The Android app uses Camera2 API for high-quality video capture, Firebase for user authentication, and WebSocket for real-time communication with the backend.
Main Components
a. Android Client (Frontend):
• Contains the mobile user interface.
• Integrates the Camera2 API for video capture.
• Uses Firebase for account management (signup, login, password recovery, email verification).
• Establishes a WebSocket connection for real-time data exchange with the backend.
b. WebSocket Communication Layer:
• Serves as the real-time communication bridge between the Android client and the backend server.
c. Backend Processing System (Server):
• Managed by various Python scripts (main.py, dumbell.py, give_angle_teacher.py, Demo.py, utils.py) that implement core functionality.
• Uses MediaPipe for detecting body landmarks and OpenCV for video/image processing.
• Incorporates machine learning models to analyze poses, count repetitions, and provide corrective feedback.
d. Data/Model Training:
• Uses CSV datasets (train_angle.csv, test_angle.csv, and yoga_Dataset/angle.csv) to train or validate angle-based pose detection models.
• Serves as a source of training data for machine learning routines integrated within the backend processing modules.
File Structure
• The presence of multiple Python scripts (e.g., main.py, dumbell.py, give_angle_teacher.py) suggests that the backend logic is modularized into separate components, possibly handling different exercise types or processing steps.
• The utility functions within utils.py likely support common functionality (data processing, model loading, etc.) across different modules.
• The CSV files and dataset directory indicate that data-driven machine learning or calibration of pose angles is an integral part of the solution.
Architectural Patterns & Design Principles
• Separation of concerns between the frontend (Android app) and backend (Python-based server).
• Modular design in the backend where specific scripts or modules handle exercises, pose detection, and feedback.
• Real-time data exchange is achieved via WebSocket, emphasizing asynchronous and event-driven communication.
• Emphasis on scalability and maintainability by separating machine learning model handling from video processing logic.
Final Elements
• Annotations or notes on key technologies (MediaPipe, OpenCV, Firebase, WebSocket, etc.) next to the corresponding components.