How shivasankari27/physiobot works
1. Purpose and runtime shape
- This is a single-process Python computer-vision application for rehab coaching.
- app.py is the orchestration entry point: it parses CLI flags, opens the camera, runs the processing loop, renders the overlay, and saves session data on exit.
2. Top-level data flow
- The main loop is a pipeline: camera frame → pose estimation → joint/angle features → scoring and safety rules → session trend tracking → on-screen overlay.
- Safety warnings are treated as a higher-priority output than normal quality feedback.
3. Pose acquisition subsystem
- pose/estimator.py is the boundary to the pose-tracking runtime, using MediaPipe skeletal landmarks.
- Its responsibility is to turn raw frames into structured landmark data for downstream analysis, not to score movement quality itself.
4. Feature extraction subsystem
- pose/features.py computes biomechanical angles from landmarks and handles left/right arm selection.
- It also supports auto-selection of the more visible arm, which matters because the rest of the system consumes a single “active arm” view.
5. Exercise configuration boundary
- config.py centralizes exercise definitions and thresholds in EXERCISES.
- Each exercise configuration defines ideal angles, danger cutoffs, scoring weights, and whether elbow scoring uses extension or flexion semantics.
- This keeps movement logic data-driven rather than hard-coded across the app.
6. Feedback and safety rules
- feedback/rules.py is the scoring and decision engine.
- It converts angle features into a 0–100 form score, generates user-facing feedback text, and detects danger zones that trigger immediate safety warnings.
- This subsystem is the policy boundary between raw kinematics and rehab guidance.
7. Session analytics and persistence
- feedback/session.py tracks scores over time, computes rolling-mean trend state, and persists the session at shutdown.
- Session records are saved as JSON under data/sessions/, capturing exercise type, timestamps, score history, average score, and trend classification.
- This module is the stateful boundary for longitudinal analysis.
8. UI presentation
- ui/overlay.py is responsible for rendering the unified on-screen status layer.
- It displays the current score, feedback message, trend, active exercise, tracked arm, and the red safety warning when applicable.
- The overlay is presentation-only; it does not own decision logic.
9. Operational modes and controls
- The app supports two exercise modes: arm_raise and bicep_curl.
- CLI flags select exercise, camera index, and arm side (left, right, or auto), which feed directly into the pose/feature and rules pipeline.
- q is the exit control that also triggers session persistence.
10. Testing and architecture coverage
- tests/test_features.py, tests/test_rules.py, and tests/test_session.py map directly to the three core logic areas: kinematic math, scoring/safety rules, and session behavior.
- The tests indicate that the repo’s important boundaries are pure logic modules around a thin real-time application shell.
11. Key architectural separation
- The repo cleanly separates acquisition, computation, policy, presentation, and persistence.
- That separation is important for diagramming: the camera and MediaPipe runtime sit at one edge, while JSON session storage sits at the other, with scoring and overlay rendering in the middle.
Main entry points: app.py App · rules.py Rules · session.py Session · features.py Features
Components 14
Runtime
CLI entrypoint and live processing loop
- App · Parses CLI flags, opens the camera, runs the real-time processing loop, renders overlays, and saves the session on exit.
app.py - Exercises · Central exercise definitions and thresholds for arm_raise and bicep_curl, including target angles, danger cutoffs, weights, and elbow semantics.
config.py - Camera · Live frame input selected by camera index.
- Exercise Mode · Selected rehab mode, either arm_raise or bicep_curl.
- Arm Side · Selected arm input, left, right, or auto.
Pose
Frame acquisition and landmark feature extraction
Feedback
Scoring, safety, and session tracking
- Rules · Maps angle features to form scores, generates user feedback, and detects danger zones for safety warnings.
feedback/rules.py - Session · Tracks score history, computes rolling trends, and persists session records as JSON at shutdown.
feedback/session.py - Session Data · Saved session records under data/sessions/.
UI
On-screen overlay rendering
- Overlay · Renders the on-screen status layer with score, feedback, trend, active exercise, tracked arm, and safety warning.
ui/overlay.py
Tests
Core logic coverage
- Feature Tests · Covers kinematic math and arm selection behavior.
tests/test_features.py - Rules Tests · Covers scoring, feedback, and danger detection behavior.
tests/test_rules.py - Session Tests · Covers session tracking, trend classification, and persistence behavior.
tests/test_session.py
Connections 18
- App → Camera: opens · Initializes the chosen camera source.
- App → Exercise Mode: selects · CLI selects the active exercise.
- App → Arm Side: selects · CLI selects left/right/auto arm mode.
- App → Estimator: feeds frames · Processing loop sends frames into pose estimation.
- Estimator → Features: landmarks · Extracted landmarks flow into feature computation.
- Features → Rules: angles · Angle features are evaluated by scoring and safety policy.
- Exercises → Rules: thresholds · Exercise definitions drive scoring and danger cutoffs.
- Exercises → Features: semantics · Exercise setup informs active arm handling and elbow interpretation.
- Rules → Session: scores · Form scores and feedback are recorded over time.
- Rules → Overlay: feedback · Normal guidance text and score are rendered in the overlay.
- Session → Overlay: trend · Rolling trend state is displayed to the user.
- Rules → Overlay: warning · Safety warnings take precedence in the visual status layer.
- Session → Session Data: persists · Shutdown writes session JSON records to disk.
- App → Overlay: renders · The main loop composes the final on-screen status layer.
- App → Session: finalizes · Exit control triggers session persistence and cleanup.
- Feature Tests → Features: covers · Validates feature extraction and arm selection logic.
- Rules Tests → Rules: covers · Validates scoring, feedback, and safety decisions.
- Session Tests → Session: covers · Validates trend tracking and persistence behavior.