# hans-hc/violina architecture

> Architecture diagram of the GitHub repository hans-hc/violina, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hans-hc/violina
- Repository: https://github.com/hans-hc/violina

## Overview

**Violina is a real-time, full-stack computer-vision posture-feedback system**:
- **Frontend (React/Vite)** captures webcam video, renders UI, and displays posture feedback/overlays in real time.
- **Backend (Flask + Python + MediaPipe)** runs pose + hand landmark models, computes geometric posture metrics, compares against a baseline, and returns feedback.
- There is also **optional hardware feedback** (Arduino buzzer) bridged via a **Node serial server**.
- The repo includes **offline/auxiliary pipelines**: baseline capture, tuning, validation, and testing assets.

> Webcam → Frame streaming → ML inference → Posture evaluation → Feedback (UI overlays + possible buzzer) → Repeat

##### File tree to architectural “containers”
From the tree, you have clear container boundaries:

##### Frontend container (`front-end/`)
Key indicators:
- `front-end/src/components/*` includes `Webcam.jsx`, `WebcamStream.jsx`, `PostureAnalysis.jsx`, `Tuner.jsx`, `Metronome.jsx`.
- Vite entrypoints: `front-end/src/main.jsx`, `front-end/src/App.jsx`, `front-end/vite.config.js`.

What to represent:
- **Webcam capture & streaming** component(s)
- **Posture feedback UI** component(s)
- **Tuning UI** (likely adjusting thresholds / sensitivity)
- **Session UX** (Home page, Navbar, CSS)

##### ML / posture backend container (`back-end/`)
Key indicators:
- Flask server entrypoints mentioned in README: `correctForm.py` and `webcam.py`.
- ML assets: `pose_landmarker.task`, `hand_landmarker.task` (MediaPipe task models).
- Core logic modules: `pose_comparator.py`, `drawingLines.py`, possibly “geometry” logic in `correctForm.py`.
- Data & artifacts:
- `baseline_images/`, `testing_images/`
- `output/*.json` (baseline & results like `baseline_data.json`, `validation_results.json`)

What to represent:
- **HTTP API** that receives frames and returns feedback.
- **Inference pipeline** (pose + hand landmark detection).
- **Posture evaluation** (comparison to baseline, rule/metric computation).
- **Overlay generation** (lines/annotations).
- **Baseline and tuning tools** (offline scripts).

##### Serial / hardware feedback container (`back-end/serial-server.js` + Arduino sketch)
Key indicators:
- `back-end/serial-server.js` (Node process to talk to serial port)
- `back-end/INCORRECT_BUZZER_SOUND/INCORRECT_BUZZER_SOUND.ino` (Arduino)

What to represent:
- **Optional alert channel**: backend/frontend triggers → Node serial bridge → Arduino buzzer.

##### Repo-level Node artifacts (`package.json`, `node_modules/`)
- Build/dev servers (Vite dev server)
- Any top-level Node service if actually executed (unclear; the explicit runtime Node service is `serial-server.js`)

##### View 1: System Context (external actors + system)
Show:
- **User (violin player)**
- **Webcam**
- **Violina System**
- Optional external: **Arduino device** (buzzer)

Purpose: communicate who uses it and what peripherals exist.

This is the “main” architecture picture.

Containers:
- **Browser (React/Vite app)**
- **Flask backend (Python)**
- **MediaPipe models** (as a dependency inside backend)
- **Baseline/Results storage** (local filesystem JSON + images)
- **Node Serial Server**
- **Arduino Buzzer**

- Browser → Flask: “Frame stream” (likely HTTP POST or WebSocket)
- Flask → Browser: “Feedback JSON + overlay instructions (lines/landmarks)”
- Flask or Browser → Node Serial: “Incorrect posture event”
- Node Serial → Arduino: “Serial command”
- Backend ↔ Filesystem: “read/write baseline_data.json, validation_results.json, baseline_images”

Split the backend into components that match the files:

- **API Layer** (in `correctForm.py` and/or `webcam.py`)
- endpoints for: frame ingestion, baseline selection, tuning params, etc. (name them after you inspect actual route definitions)
- **Inference Layer**
- MediaPipe Pose Landmarker (`pose_landmarker.task`)
- MediaPipe Hand Landmarker (`hand_landmarker.task`)
- **Posture Analysis Layer**
- `pose_comparator.py` (baseline comparison)
- `drawingLines.py` (overlay geometry)
- any metric/rule computation in `correctForm.py`
- **Baseline & Validation Tooling**
- `baseline_collector.py`, `run_baseline_collection.py`
- `validate_poses.py`
- `tuning.py`, `tuning_requirements.txt`
- **Artifacts**
- `output/*.json`, `baseline_images/`, `testing_images/`

API → Inference → Analysis → Response builder → API response.

##### Produce
- posture status (ok / drifting / incorrect)
- suggestions
- overlay primitives (lines/points)
##### If incorrect: trigger buzzer event path (optional)

Given the README + tree, ensure these interactions are not vague:

##### Webcam and streaming boundary
- The webcam is accessed in the **browser**, but “streaming frames” crosses to the backend.
- This as a **high-frequency call path**; annotate “low latency, many requests/sec” or “WebSocket stream” (after code confirmation).

##### Baseline lifecycle
There’s a baseline pipeline indicated by:
- `baseline_collector.py`, `run_baseline_collection.py`, `baseline_images/`, `output/baseline_data.json`

- Baseline acquisition (offline or guided mode) → stored JSON/images → used by live comparator.

##### Model assets as deployable dependencies
The `.task` files are deployable artifacts and should be shown as:
- “Model files packaged with backend image/runtime”

##### Optional alerting hardware path
Keep it separate with a clear “optional” label:
- Posture incorrect event → serial server → Arduino buzzer

#### 5) Diagramming guidelines tailored to this repo (be very detailed; split into many components)
##### Component naming: match the code
Use the actual file/module names to avoid abstraction drift:
- `correctForm.py` = “Flask API + posture evaluation orchestration”
- `webcam.py` = “webcam/frame source utility or streaming server” (confirm role in code)
- `pose_comparator.py` = “baseline comparison logic”
- `drawingLines.py` = “overlay primitives / landmark visualization”
- `baseline_collector.py` = “baseline capture pipeline”
- `serial-server.js` = “serial bridge service”

##### Distinguish runtime vs offline tools
In the backend, visually separate:
- **Runtime services** (Flask API, inference)
- **Offline scripts** (baseline collection, validation, tuning)

This set is detailed, matches the repo structure, and separates concerns without overthinking it—while still breaking the system into as many concrete components as possible.

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =======================
%% View 1: System Context
%% =======================
subgraph "View 1 — System Context"
direction TB

U["User (Violin player)"]:::external
Cam["Webcam (Browser getUserMedia)"]:::external
Violina["Violina System"]:::system
Arduino["Arduino Buzzer (Optional)"]:::hardware

U -->|"plays_violin"| Cam
U -->|"views_feedback"| Violina
Cam -->|"video_frames"| Violina
Violina -.->|"audible_alert(optional)"| Arduino
end

%% =======================
%% View 2: Container Diagram
%% =======================
subgraph "View 2 — Container Diagram (Runtime)"
direction TB

subgraph "Browser (React/Vite Frontend)"
direction TB
FE["Frontend Container"]:::frontend
App["React App (App.jsx)"]:::frontend
Main["Bootstrap (main.jsx)"]:::frontend
Vite["Vite Config"]:::frontend
WebcamC["Webcam UI (Webcam.jsx)"]:::frontend
WebcamStream["Frame Streaming (WebcamStream.jsx)"]:::frontend
PostureUI["Posture Feedback UI (PostureAnalysis.jsx)"]:::frontend
TunerUI["Tuning UI (Tuner.jsx)"]:::frontend
Nav["Navbar (Navbar.jsx)"]:::frontend
Home["Home Page (Home.jsx)"]:::frontend
Metro["Metronome (Metronome.jsx)"]:::frontend
Styles["Styles (App.css + styles/)"]:::frontend
end

subgraph "Python Backend (Flask + MediaPipe)"
direction TB
BE["Backend Container"]:::backend
API["Flask API Orchestrator (correctForm.py)"]:::api
WebcamBE["Backend Webcam/Stream Utility (webcam.py)"]:::backend
Req["Python Deps (requirements.txt)"]:::backend

subgraph "MediaPipe Inference Assets"
direction TB
PoseTask["Pose Landmarker Model (.task)"]:::model
HandTask["Hand Landmarker Model (.task)"]:::model
end

FS[("Local Filesystem Artifacts\n(JSON + Images)")]:::storage
end

subgraph "Optional Hardware Feedback"
direction TB
SerialSrv["Node Serial Bridge (serial-server.js)"]:::hardware
Buzzer["Arduino Sketch (Buzzer)"]:::hardware
end

%% Runtime connections (container-level)
WebcamStream ==>|"frame_stream(image_frame:JPEG/base64/bytes)\n(high_frequency)"| API
API ==>|"feedback_JSON{postureScore,warnings,overlayLines}"| PostureUI

API -->|"loads_models"| PoseTask
API -->|"loads_models"| HandTask

API <-->|"read/write\nbaseline_data.json,results.json,images"| FS

%% Optional buzzer path (intent-level)
API -.->|"incorrect_posture_event(optional)"| SerialSrv
PostureUI -.->|"incorrect_posture_event(optional)"| SerialSrv
SerialSrv -->|"serial_command"| Buzzer
end

%% =======================
%% View 3: Backend Component Diagram
%% =======================
subgraph "View 3 — Backend Component Diagram (back-end/)"
direction TB

subgraph "Runtime Path (per-frame)"
direction TB
API3["API Layer (correctForm.py)"]:::api
Ingest["Frame Ingestion\n(decode/normalize)"]:::backend
Infer["Inference Layer\n(MediaPipe Pose+Hands)"]:::backend
Comparator["Posture Comparison (pose_comparator.py)"]:::backend
Overlay["Overlay Primitives (drawingLines.py)"]:::backend
Response["Response Builder\n(JSON feedback + overlay instructions)"]:::backend
end

subgraph "Deployable Model Files"
direction TB
PoseTask3["pose_landmarker.task"]:::model
HandTask3["hand_landmarker.task"]:::model
end

subgraph "Artifacts (Local FS)"
direction TB
OutDir[("output/ (results)")]:::storage
BaselineJSON[("baseline_data.json")]:::storage
ValidationJSON[("validation_results.json")]:::storage
ResultJSON[("result.json")]:::storage
TestJSON[("test.json")]:::storage
BaselineImgs[("baseline_images/")]:::storage
TestingImgs[("testing_images/")]:::storage
end

subgraph "Offline / Auxiliary Tooling"
direction TB
BaseCollect["baseline_collector.py"]:::tooling
RunBase["run_baseline_collection.py"]:::tooling
Validate["validate_poses.py"]:::tooling
Tune["tuning.py"]:::tooling
TuneReq["tuning_requirements.txt"]:::tooling
end

%% Component connections (runtime)
API3 -->|"receives_frame"| Ingest
Ingest -->|"image_tensor"| Infer
Infer -->|"pose_landmarks + hand_landmarks"| Comparator
Comparator -->|"posture_metrics + status"| Overlay
Overlay -->|"overlay_lines/points"| Response
Response -->|"HTTP_response(JSON)"| API3

Infer -.->|"uses"| PoseTask3
Infer -.->|"uses"| HandTask3

Comparator <-->|"loads_baseline"| BaselineJSON
Overlay -->|"may_log"| OutDir

%% Tooling connections (offline)
RunBase -->|"runs"| BaseCollect
BaseCollect -->|"writes"| BaselineImgs
BaseCollect -->|"writes"| BaselineJSON
Validate -->|"reads"| TestingImgs
Validate -->|"reads"| BaselineJSON
Validate -->|"writes"| ValidationJSON
Tune -->|"reads/writes_thresholds(optional)"| OutDir
Tune -.->|"uses"| TuneReq

OutDir -->|"contains"| ResultJSON
OutDir -->|"contains"| TestJSON
OutDir -->|"contains"| ValidationJSON
OutDir -->|"contains"| BaselineJSON
end

%% =======================
%% View 4: Real-time Sequence (single frame)
%% =======================
subgraph "View 4 — Real-time Feedback Loop (Sequence)"
direction TB

subgraph "Frontend (Browser)"
direction TB
S1["WebcamStream.jsx\ncapture_frame()"]:::frontend
S2["send_frame() to backend"]:::frontend
S7["PostureAnalysis.jsx\nrender_overlay() + UI hints"]:::frontend
S8["Optional: trigger_buzzer_event()"]:::frontend
end

subgraph "Backend (Flask)"
direction TB
S3["correctForm.py\nreceive_frame"]:::api
S4["MediaPipe\npose+hand inference"]:::backend
S5["Compute metrics\ncompare_to_baseline"]:::backend
S6["Build feedback\nJSON overlay primitives"]:::backend
end

subgraph "Optional Hardware"
direction TB
S9["serial-server.js\nwrite_serial(cmd)"]:::hardware
S10["Arduino\nbuzzer_sound()"]:::hardware
end

S1 -->|"frame(image)"| S2
S2 ==>|"HTTP_request(high_frequency)"| S3
S3 -->|"decode_frame"| S4
S4 -->|"landmarks"| S5
S5 -->|"status+suggestions+overlay"| S6
S6 ==>|"HTTP_response(JSON)"| S7
S7 -.->|"if_incorrect"| S8
S8 -.->|"event(optional)"| S9
S9 -->|"serial_cmd"| S10
S7 -->|"repeat_next_frame"| S1
end

%% =======================
%% Click Events (from component mapping)
%% =======================
click FE "https://github.com/hans-hc/violina/tree/main/front-end/"
click Main "https://github.com/hans-hc/violina/blob/main/front-end/src/main.jsx"
click App "https://github.com/hans-hc/violina/blob/main/front-end/src/App.jsx"
click Vite "https://github.com/hans-hc/violina/blob/main/front-end/vite.config.js"
click WebcamC "https://github.com/hans-hc/violina/blob/main/front-end/src/components/Webcam.jsx"
click WebcamStream "https://github.com/hans-hc/violina/blob/main/front-end/src/components/WebcamStream.jsx"
click PostureUI "https://github.com/hans-hc/violina/blob/main/front-end/src/components/PostureAnalysis.jsx"
click TunerUI "https://github.com/hans-hc/violina/blob/main/front-end/src/components/Tuner.jsx"
click Home "https://github.com/hans-hc/violina/blob/main/front-end/src/pages/Home.jsx"
click Nav "https://github.com/hans-hc/violina/blob/main/front-end/src/components/Navbar.jsx"
click Metro "https://github.com/hans-hc/violina/blob/main/front-end/src/components/Metronome.jsx"
click Styles "https://github.com/hans-hc/violina/tree/main/front-end/src/styles/"
click BE "https://github.com/hans-hc/violina/tree/main/back-end/"
click API "https://github.com/hans-hc/violina/blob/main/back-end/correctForm.py"
click API3 "https://github.com/hans-hc/violina/blob/main/back-end/correctForm.py"
click WebcamBE "https://github.com/hans-hc/violina/blob/main/back-end/webcam.py"
click PoseTask "https://github.com/hans-hc/violina/blob/main/back-end/pose_landmarker.task"
click HandTask "https://github.com/hans-hc/violina/blob/main/back-end/hand_landmarker.task"
click PoseTask3 "https://github.com/hans-hc/violina/blob/main/back-end/pose_landmarker.task"
click HandTask3 "https://github.com/hans-hc/violina/blob/main/back-end/hand_landmarker.task"
click Comparator "https://github.com/hans-hc/violina/blob/main/back-end/pose_comparator.py"
click Overlay "https://github.com/hans-hc/violina/blob/main/back-end/drawingLines.py"
click Req "https://github.com/hans-hc/violina/blob/main/back-end/requirements.txt"
click BaseCollect "https://github.com/hans-hc/violina/blob/main/back-end/baseline_collector.py"
click RunBase "https://github.com/hans-hc/violina/blob/main/back-end/run_baseline_collection.py"
click BaselineImgs "https://github.com/hans-hc/violina/tree/main/back-end/baseline_images/"
click Validate "https://github.com/hans-hc/violina/blob/main/back-end/validate_poses.py"
click TestingImgs "https://github.com/hans-hc/violina/tree/main/back-end/testing_images/"
click Tune "https://github.com/hans-hc/violina/blob/main/back-end/tuning.py"
click TuneReq "https://github.com/hans-hc/violina/blob/main/back-end/tuning_requirements.txt"
click OutDir "https://github.com/hans-hc/violina/tree/main/back-end/output/"
click BaselineJSON "https://github.com/hans-hc/violina/blob/main/back-end/output/baseline_data.json"
click ValidationJSON "https://github.com/hans-hc/violina/blob/main/back-end/output/validation_results.json"
click ResultJSON "https://github.com/hans-hc/violina/blob/main/back-end/output/result.json"
click TestJSON "https://github.com/hans-hc/violina/blob/main/back-end/output/test.json"
click SerialSrv "https://github.com/hans-hc/violina/blob/main/back-end/serial-server.js"
click Buzzer "https://github.com/hans-hc/violina/blob/main/back-end/INCORRECT_BUZZER_SOUND/INCORRECT_BUZZER_SOUND.ino"

%% =======================
%% Styles
%% =======================
classDef external fill:#F5F5F5,stroke:#616161,stroke-width:1px,color:#212121;
classDef system fill:#E3F2FD,stroke:#1565C0,stroke-width:2px,color:#0D47A1;
classDef frontend fill:#E8F5E9,stroke:#2E7D32,stroke-width:2px,color:#1B5E20;
classDef backend fill:#FFF3E0,stroke:#EF6C00,stroke-width:2px,color:#E65100;
classDef api fill:#FFE0B2,stroke:#E65100,stroke-width:3px,color:#BF360C;
classDef model fill:#F3E5F5,stroke:#6A1B9A,stroke-width:2px,color:#4A148C,stroke-dasharray: 5 4;
classDef storage fill:#E0F7FA,stroke:#00838F,stroke-width:2px,color:#006064;
classDef tooling fill:#FFFDE7,stroke:#F9A825,stroke-width:2px,color:#F57F17,stroke-dasharray: 4 3;
classDef hardware fill:#FFEBEE,stroke:#C62828,stroke-width:2px,color:#B71C1C;
```
