How kailun1103/ai_coach_detection works
Project Type and High-Level Purpose
• This is a full-stack, computer-vision/ML-powered sports-analytics web application.
• Key flows: Video capture → CV/3D processing → ML feedback → Web visualization.
Break Down the File Structure into Architectural Components
• “camera_control(FASTAPI)_*.py” and “server.js” + “package.json”/“node_modules” → Backend API & Web server.
• “binocular_correction/…”, “trajector_*”, “video_*”, “trajectory_*” → Processing pipeline (calibration, pose/ball detection, smoothing, sync, 3D triangulation).
• “model/… .pt” + “single_feedback/…” + “trajectory_gpt*.py” → ML models & feedback engine.
• “drawing_*.html/js/css”, “main.js” → Frontend UI (Three.js, Plotly, control panel).
• “play_records/…” → Data storage of swing JSONs (could be flat-file or DB).
Main Components
a. Video Capture Layer
• Dual cameras (side & 45°) feed into the system.
• Modules: camera_control(FASTAPI).py or main.py triggers capture.
b. Preprocessing & Synchronization
• video_clip.py, video_sync.py, pic_cut.py
• FFmpeg invoked via scripts.
c. Computer-Vision & Trajectory Pipeline
• YOLOv8 pose detection (yolov8n-pose.pt)
• Tennis ball detection (custom tennisball_OD_v1.pt)
• 2D keypoint extraction (trajectory_2D_output.py), smoothing (trajector_2D_smoothing.py)
• Binocular correction & 3D triangulation (binocular_correction/*.py)
d. Feedback Engine
• KNN analysis (trajectory_knn.py)
• GPT-powered feedback (trajectory_gpt*.py, single_feedback/prompt.py)
e. Data Layer
• play_records/ JSON files
• Model files in “model/”
f. Web Server & API
• FastAPI (camera_control*.py) or Express.js (server.js)
• Exposes REST endpoints for: upload video, start analysis, fetch results
g. Frontend Web Client
• main.js, drawing_3D_three_js.html, drawing_2D_chart_js.html
• Uses Three.js for 3D, Plotly for charts
• Calls backend APIs and renders results
Interactions & Data Flow
• External services: language-model API (e.g., OpenAI GPT).
• Model loading: loaded by backend modules at startup.
Architectural Patterns
• Modular pipeline: each phase encapsulated in its own directory/script.
• Microservices style: separate capture API vs. analysis engine vs. web server.
• MVC-like separation: “server.js”/FastAPI = Controller, data files = Model, “main.js” = View.
Tool Recommendations
• Export as SVG/PNG for inclusion in docs.