How nmhp16/motion_ai works
Project type and purpose
• Motion AI is a hybrid desktop-plus-microservice application.
• It uses a JavaFX front end (GUI), a set of Python modules (pose estimation, real-time processing, AI calls), and a local “motion_database” of professional dance videos/key-point data.
File structure
• Top-level “src/main/java/com/instructor/…” contains your core Java backend:
• algorithms (DTW, sorts), data readers/processors, evaluators (scoring/feedback), controller & UI entry points.
• “motion_database” holds raw video files and annotation text.
• “pose_detection” and “real_time” are Python scripts using MediaPipe + OpenCV.
• “web_call” houses a Python module for external AI service calls (SpeechRecognition/PyAudio).
• POM.xml and module-info.java define Maven/JavaFX configuration.
Extract architectural insights from the README
• JavaFX UI (“DanceInstructorUI”) orchestrates the user flow.
• Python scripts perform pose estimation (real time or batch), returning keypoints maps (JSON/text).
• Core algorithms (MergeSort, DTW, Normalization, Interpolation, Moving Average) run in Java under “com.instructor.algorithms” and “com.instructor.data” → “com.instructor.evaluation.”
• Data flows from raw video → Python MediaPipe → Java processing → UI feedback.
Main components and their roles
A. Frontend / Presentation layer
• JavaFX GUI (DanceInstructorUI + controller)
B. Backend Java processing layer
• FileCleanup/ApplicationHandler (I/O)
• PoseDataReader/Processor → sorts, normalizes, interpolates → evaluator
• Algorithms (DTW, sorts)
• JointScoring/PoseFeedback
C. Python services
1. pose_detection/PoseDetection.py & PoseUpload.py – offline pose extraction
2. real_time/app.py – real-time pose stream (Flask or similar)
3. web_call/AI_Call.py – optional external AI/Speech service
D. Data storage
• motion_database (AVI + TXT)
• professional_keypoints.json (real_time)
E. External services & libs
• MediaPipe, OpenCV, SpeechRecognition, PyAudio
Relationships & data flows
1. User interacts with JavaFX UI → loads video or selects real-time mode
2. UI invokes Python pose service (batch or real time)
• Could be via subprocess calls (e.g. “python PoseDetection.py”) or HTTP API calls to real_time/app.py
3. Python returns keypoints JSON/text → UI/backend layer
4. Java processing pipeline:
a. PoseDataReader parses JSON/text
b. MergeSort/InsertionSort orders frames
c. DTW aligns user vs. pro sequences
d. Normalization (torso scaling)
e. Linear interpolation fills gaps
f. Moving average smooths data
g. JointScoring & PoseFeedback compute metrics
5. UI displays feedback, scores, and suggestions
6. Optionally, UI or backend calls AI_Call.py for voice guidance
Key architectural patterns & principles
• Layered architecture: Presentation → Application logic → Data processing → Data storage
• Microservices style for Python pose modules (separate processes or APIs)
• Modular algorithms (each in its own class under com.instructor.algorithms)
• File-based data store (no heavy DB) for videos/keypoints
• Major components and label them.
• Related classes under package-level containers.
• File-storage nodes for motion_database and JSON outputs.