# kailun1103/ai_coach_detection architecture

> Architecture diagram of the GitHub repository kailun1103/ai_coach_detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kailun1103/ai_coach_detection
- Repository: https://github.com/kailun1103/ai_coach_detection

## Overview

##### Project Type and High-Level Purpose
- This is a full-stack, computer-vision/ML-powered sports-analytics web application.
- Key flows: Video capture → CV/3D processing → ML feedback → Web visualization.

##### Break Down the File Structure into Architectural Components
- “camera_control(FASTAPI)_*.py” and “server.js” + “package.json”/“node_modules” → Backend API & Web server.
- “binocular_correction/…”, “trajector_*”, “video_*”, “trajectory_*” → Processing pipeline (calibration, pose/ball detection, smoothing, sync, 3D triangulation).
- “model/… .pt” + “single_feedback/…” + “trajectory_gpt*.py” → ML models & feedback engine.
- “drawing_*.html/js/css”, “main.js” → Frontend UI (Three.js, Plotly, control panel).
- “play_records/…” → Data storage of swing JSONs (could be flat-file or DB).

##### Main Components
a. Video Capture Layer
- Dual cameras (side & 45°) feed into the system.
- Modules: camera_control(FASTAPI).py or main.py triggers capture.
b. Preprocessing & Synchronization
- video_clip.py, video_sync.py, pic_cut.py
- FFmpeg invoked via scripts.
c. Computer-Vision & Trajectory Pipeline
- YOLOv8 pose detection (yolov8n-pose.pt)
- Tennis ball detection (custom tennisball_OD_v1.pt)
- 2D keypoint extraction (trajectory_2D_output.py), smoothing (trajector_2D_smoothing.py)
- Binocular correction & 3D triangulation (binocular_correction/*.py)
d. Feedback Engine
- KNN analysis (trajectory_knn.py)
- GPT-powered feedback (trajectory_gpt*.py, single_feedback/prompt.py)
e. Data Layer
- play_records/ JSON files
- Model files in “model/”
f. Web Server & API
- FastAPI (camera_control*.py) or Express.js (server.js)
- Exposes REST endpoints for: upload video, start analysis, fetch results
g. Frontend Web Client
- main.js, drawing_3D_three_js.html, drawing_2D_chart_js.html
- Uses Three.js for 3D, Plotly for charts
- Calls backend APIs and renders results

##### Interactions & Data Flow
- External services: language-model API (e.g., OpenAI GPT).
- Model loading: loaded by backend modules at startup.

##### Architectural Patterns
- Modular pipeline: each phase encapsulated in its own directory/script.
- Microservices style: separate capture API vs. analysis engine vs. web server.
- MVC-like separation: “server.js”/FastAPI = Controller, data files = Model, “main.js” = View.

##### Tool Recommendations
- Export as SVG/PNG for inclusion in docs.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Capture Layer
    subgraph "Video Capture Layer"
        direction TB
        CC1["camera_control FASTAPI (3253).py"]:::python
        CC2["camera_control FASTAPI (9436).py"]:::python
        ORCH["Capture Orchestrator (main.py)"]:::python
    end

    %% Preprocessing & Synchronization
    subgraph "Preprocessing & Synchronization"
        direction TB
        VC["tool/video_clip.py"]:::python
        VCSYNC["tool/video_clip_sync.py"]:::python
        VM["tool/video_metadata.py"]:::python
        VMSYNC["tool/video_metadata_sync.py"]:::python
        VMERGE["video_merge.py"]:::python
        VSYNC["video_sync.py"]:::python
        PC1["binocular_correction/pic_cut.py"]:::python
        PC2["binocular_correction/pic_cut_2.py"]:::python
    end

    %% Computer-Vision & Trajectory Pipeline
    subgraph "Computer-Vision & Trajectory Pipeline"
        direction TB
        DET["video_detection.py"]:::python
        T2D["trajectory_2D_output.py"]:::python
        S2D["trajector_2D_smoothing.py"]:::python
        R2D["trajector_2D_capture_swing_range.py"]:::python
        T3D["trajectory_3D_output.py"]:::python
        S3D["trajector_3D_smoothing.py"]:::python
        R3D["trajector_3D_capture_swing_range.py"]:::python
        BC["binocular_correction/binocular_correction.py"]:::python
        DCB["binocular_correction/draw_chessboard.py"]:::python
    end

    %% Feedback Engine
    subgraph "Feedback Engine"
        direction TB
        KNN["trajectory_knn.py"]:::ml
        GPT1["trajectory_gpt.py"]:::ml
        GPT2["trajectory_gpt_overall_feedback.py"]:::ml
        GPT3["trajectory_gpt_single_feedback.py"]:::ml
        PROMPT["single_feedback/prompt.py"]:::ml
        MCONFIG["single_feedback/model_config.py"]:::ml
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        RECORDS["Swing Records (play_records/*.json)"]:::data
        MODELS["ML Models (model/*.pt)"]:::data
    end

    %% Web Server & API
    subgraph "Web Server & API"
        direction TB
        EXPRESS["Express.js Server (server.js)"]:::api
        PKG["package.json & lock"]:::api
        NODES["node_modules/"]:::api
    end

    %% Frontend Web Client
    subgraph "Frontend Web Client"
        direction TB
        UI["main.js"]:::frontend
        D3D["drawing_3D_three_js.html"]:::frontend
        P3D["drawing_3D_plotly.py"]:::frontend
        D2D["drawing_2D_chart_js.html"]:::frontend
        P2D["drawing_2D_plotly.py"]:::frontend
        CSS["drawing_2D_chart_js.css"]:::frontend
        ARCH["2d_html/.archive/"]:::frontend
        RDMD["2d_html/README.MD"]:::frontend
    end

    %% External Services
    GPTAPI["GPT External API"]:::external

    %% Data Flows
    Cameras["Dual Cameras"]:::external

    Cameras --> ORCH
    ORCH -->|trigger| VC
    VC --> VCSYNC
    VCSYNC --> VM
    VM --> VMSYNC
    VMSYNC --> VMERGE
    VMERGE --> VSYNC
    VSYNC --> PC1
    PC1 --> PC2

    PC2 --> DET
    DET --> T2D
    T2D --> S2D
    S2D --> R2D
    R2D --> T3D
    T3D --> S3D
    S3D --> R3D
    BC --> DCB
    DCB --> T3D
    R3D --> KNN
    R3D --> GPT1

    KNN -->|local analysis| RECORDS
    GPT1 -->|calls| GPTAPI
    GPT1 --> RECORDS
    GPT2 -->|calls| GPTAPI
    GPT3 -->|calls| GPTAPI
    PROMPT --> GPT1
    MCONFIG --> GPT1

    RECORDS --> EXPRESS
    MODELS --> DET
    MODELS --> KNN
    MODELS --> GPT1

    EXPRESS -->|REST: POST /analyze| UI
    EXPRESS -->|REST: GET /trajectory| D3D
    EXPRESS --> UI
    EXPRESS --> D2D

    %% Click Events
    click CC1 "https://github.com/kailun1103/ai_coach_detection/blob/prd/camera_control(FASTAPI)_3253.py"
    click CC2 "https://github.com/kailun1103/ai_coach_detection/blob/prd/camera_control(FASTAPI)_9436.py"
    click ORCH "https://github.com/kailun1103/ai_coach_detection/blob/prd/main.py"
    click VC "https://github.com/kailun1103/ai_coach_detection/blob/prd/tool/video_clip.py"
    click VCSYNC "https://github.com/kailun1103/ai_coach_detection/blob/prd/tool/video_clip_sync.py"
    click VM "https://github.com/kailun1103/ai_coach_detection/blob/prd/tool/video_metadata.py"
    click VMSYNC "https://github.com/kailun1103/ai_coach_detection/blob/prd/tool/video_metadata_sync.py"
    click VMERGE "https://github.com/kailun1103/ai_coach_detection/blob/prd/video_merge.py"
    click VSYNC "https://github.com/kailun1103/ai_coach_detection/blob/prd/video_sync.py"
    click PC1 "https://github.com/kailun1103/ai_coach_detection/blob/prd/binocular_correction/pic_cut.py"
    click PC2 "https://github.com/kailun1103/ai_coach_detection/blob/prd/binocular_correction/pic_cut_2.py"
    click DET "https://github.com/kailun1103/ai_coach_detection/blob/prd/video_detection.py"
    click T2D "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajectory_2D_output.py"
    click S2D "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajector_2D_smoothing.py"
    click R2D "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajector_2D_capture_swing_range.py"
    click T3D "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajectory_3D_output.py"
    click S3D "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajector_3D_smoothing.py"
    click R3D "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajector_3D_capture_swing_range.py"
    click BC "https://github.com/kailun1103/ai_coach_detection/blob/prd/binocular_correction/binocular_correction.py"
    click DCB "https://github.com/kailun1103/ai_coach_detection/blob/prd/binocular_correction/draw_chessboard.py"
    click KNN "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajectory_knn.py"
    click GPT1 "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajectory_gpt.py"
    click GPT2 "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajectory_gpt_overall_feedback.py"
    click GPT3 "https://github.com/kailun1103/ai_coach_detection/blob/prd/trajectory_gpt_single_feedback.py"
    click PROMPT "https://github.com/kailun1103/ai_coach_detection/blob/prd/single_feedback/prompt.py"
    click MCONFIG "https://github.com/kailun1103/ai_coach_detection/blob/prd/single_feedback/model_config.py"
    click EXPRESS "https://github.com/kailun1103/ai_coach_detection/blob/prd/server.js"
    click PKG "https://github.com/kailun1103/ai_coach_detection/blob/prd/package.json"
    click NODES "https://github.com/kailun1103/ai_coach_detection/tree/prd/node_modules/"
    click UI "https://github.com/kailun1103/ai_coach_detection/blob/prd/main.js"
    click D3D "https://github.com/kailun1103/ai_coach_detection/blob/prd/drawing_3D_three_js.html"
    click P3D "https://github.com/kailun1103/ai_coach_detection/blob/prd/drawing_3D_plotly.py"
    click D2D "https://github.com/kailun1103/ai_coach_detection/blob/prd/drawing_2D_chart_js.html"
    click P2D "https://github.com/kailun1103/ai_coach_detection/blob/prd/drawing_2D_plotly.py"
    click CSS "https://github.com/kailun1103/ai_coach_detection/blob/prd/drawing_2D_chart_js.css"
    click ARCH "https://github.com/kailun1103/ai_coach_detection/tree/prd/2d_html/.archive/"
    click RDMD "https://github.com/kailun1103/ai_coach_detection/blob/prd/2d_html/README.MD"

    %% Styles
    classDef python fill:#a8f0c6,stroke:#333,stroke-width:1px
    classDef ml fill:#f9e79f,stroke:#333,stroke-width:1px
    classDef api fill:#aed6f1,stroke:#333,stroke-width:1px
    classDef frontend fill:#fad7a0,stroke:#333,stroke-width:1px
    classDef data fill:#d5dbdb,stroke:#333,stroke-width:1px
    classDef external fill:#f5b7b1,stroke:#333,stroke-width:1px
```
