# thanksmors/pbvision architecture

> Architecture diagram of the GitHub repository thanksmors/pbvision, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thanksmors/pbvision
- Repository: https://github.com/thanksmors/pbvision

## Overview

1. **Repository shape**
   - The repo is split into a portable analysis engine under `engine/pbengine/` and a thin application layer under `api/`.
   - The engine can run standalone from the command line to produce `result.json`; the API wraps that engine for uploads and local viewing.

2. **End-to-end purpose**
   - The system analyzes a pickleball match video at the point level.
   - The main output is a structured JSON result containing rallies/points, serve and bounce events, winner calls, and confidence / review flags.

3. **Top-level data flow**
   - The pipeline is: video probe → court detection/homography → player tracking → ball tracking → rally segmentation → serve/bounce/winner inference → JSON export.
   - The key architectural boundary is between video/ML-heavy perception stages and the deterministic logic stages that operate on rectified coordinates and tracked events.

4. **Engine subsystem**
   - `engine/pbengine/pipeline.py` is the orchestration entry point.
   - It ties together IO, court, player, ball, and rally stages and is designed to run both with real models and with synthetic fixtures.
   - Model-backed stages are dependency-injected and can be swapped for test fixtures, which is why the engine is usable without weights for demos and tests.

5. **Court geometry subsystem**
   - `court/detector.py`, `court/homography.py`, and `court/court_model.py` handle court finding and coordinate normalization.
   - This subsystem maps camera-space detections into a normalized pickleball court frame, which is crucial because later heuristics assume a stable court coordinate system.

6. **Player tracking subsystem**
   - `detect/players.py` uses Ultralytics YOLO26 plus ByteTrack.
   - Its role is to identify and track players across frames, feeding contextual signals into later rally and serve logic rather than directly producing the final point result.

7. **Ball tracking subsystem**
   - `ball/tracker.py` and `ball/kalman.py` handle the highest-risk stage: locating the ball and smoothing its trajectory.
   - The README notes this stage uses WASB-SBDT plus jump gating and Kalman smoothing, and the architecture treats it as the most failure-prone component.
   - Output from this stage is a rectified ball path used by the rally logic.

8. **Rally / point logic subsystem**
   - `rally/segmentation.py`, `rally/serve.py`, and `rally/winner.py` convert the tracked trajectory into point boundaries and outcomes.
   - This layer is mostly heuristic and deterministic: it decides where rallies start/end, how serves are identified, and who won each rally.
   - `bounce/heuristic.py` supports bounce inference, which is a core input to winner classification.

9. **Schema and contract**
   - `schema/models.py` defines the structured result contract used by both the engine and the API/viewer.
   - The JSON schema is the integration boundary: the engine writes it, and the app/front end renders it.
   - This contract is what allows the demo fixture pipeline, CLI pipeline, and UI to share one representation.

10. **API and viewer**
   - `api/app/main.py` exposes the FastAPI service; `api/app/jobs.py` manages job execution/state.
   - The app does not appear to run the ML pipeline inline; instead it spawns the engine as a subprocess and polls a status file, which keeps the web layer thin and isolates the analysis runtime.
   - `api/frontend/index.html` is a single-page viewer for displaying the uploaded video and per-point analysis results.

11. **Fixtures and synthetic demo path**
   - `fixtures.py` provides scripted stand-ins for the model-backed stages.
   - This enables a fully CPU-only demo run and exercises the full pipeline, including segmentation and winner reasoning, without requiring model weights or real footage.
   - The tests also lean on this boundary to validate logic independently of ML inference.

12. **Testing and support tooling**
   - `engine/tests/` covers the pure-logic pieces: bounce, homography, Kalman, rally, schema, winner, and fixtures.
   - `scripts/download_weights.sh` and `scripts/run_local.sh` support two modes of use: local app development and model-backed analysis with downloaded weights.
   - `pyproject.toml` likely separates core, API, ML, and dev dependencies, matching the engine/app boundary.

13. **Architectural takeaway**
   - The repo is intentionally organized around a “portable engine + thin UI wrapper” pattern.
   - All expensive or uncertain perception work is isolated in the engine, while the API and frontend only coordinate runs and present the resulting point-level analysis.

Main entry points: [`engine/pbengine/pipeline.py`](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/pipeline.py) (Pipeline), [`engine/pbengine/rally/segmentation.py`](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/rally/segmentation.py) (Rally split), [`engine/pbengine/fixtures.py`](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/fixtures.py) (Fixtures), [`api/app/jobs.py`](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/api/app/jobs.py) (Job state)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_engine["Engine"]
  node_pipeline["Pipeline<br/>engine orchestrator<br/>[pipeline.py]"]
  node_video_io["Video probe<br/>media IO<br/>[video.py]"]
  node_court_detector["Court detect<br/>court finding<br/>[detector.py]"]
  node_homography["Homography<br/>rectification<br/>[homography.py]"]
  node_court_model["Court model<br/>court geometry<br/>[court_model.py]"]
  node_players["Player track<br/>YOLO + ByteTrack<br/>[players.py]"]
  node_ball_tracker["Ball track<br/>ball localization<br/>[tracker.py]"]
  node_kalman["Kalman<br/>trajectory smoothing<br/>[kalman.py]"]
  node_segmentation["Rally split<br/>point segmentation<br/>[segmentation.py]"]
  node_serve["Serve logic<br/>serve heuristic<br/>[serve.py]"]
  node_bounce["Bounce heuristic<br/>bounce inference<br/>[heuristic.py]"]
  node_winner["Winner logic<br/>point outcome<br/>[winner.py]"]
end

subgraph group_api["API / UI"]
  node_api_main["API app<br/>FastAPI entrypoint<br/>[main.py]"]
  node_api_jobs["Job state<br/>run coordinator<br/>[jobs.py]"]
  node_frontend["Viewer<br/>single-page UI<br/>[index.html]"]
end

subgraph group_support["Support"]
  node_schema["Result schema<br/>JSON contract<br/>[models.py]"]
  node_fixtures["Fixtures<br/>synthetic stages<br/>[fixtures.py]"]
end

node_api_main -->|"delegates runs"| node_api_jobs
node_api_jobs -->|"launches engine"| node_pipeline
node_api_main -->|"serves UI"| node_frontend
node_pipeline -->|"reads video"| node_video_io
node_pipeline -->|"detects court"| node_court_detector
node_court_detector -->|"feeds geometry"| node_homography
node_homography -->|"maps to court"| node_court_model
node_pipeline -->|"tracks players"| node_players
node_pipeline -->|"tracks ball"| node_ball_tracker
node_ball_tracker -->|"smooths path"| node_kalman
node_pipeline -->|"splits rallies"| node_segmentation
node_segmentation -->|"identifies serves"| node_serve
node_segmentation -->|"infers bounces"| node_bounce
node_segmentation -->|"classifies outcome"| node_winner
node_bounce -->|"supports decisions"| node_winner
node_pipeline -->|"writes result"| node_schema
node_fixtures -->|"swap stages"| node_pipeline
node_fixtures -->|"drives demos"| node_segmentation
node_fixtures -->|"drives demos"| node_winner
node_api_jobs -->|"reads result"| node_schema
node_frontend -->|"renders result"| node_schema

click node_api_main "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/api/app/main.py"
click node_api_jobs "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/api/app/jobs.py"
click node_frontend "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/api/frontend/index.html"
click node_pipeline "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/pipeline.py"
click node_video_io "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/io/video.py"
click node_court_detector "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/court/detector.py"
click node_homography "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/court/homography.py"
click node_court_model "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/court/court_model.py"
click node_players "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/detect/players.py"
click node_ball_tracker "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/ball/tracker.py"
click node_kalman "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/ball/kalman.py"
click node_segmentation "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/rally/segmentation.py"
click node_serve "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/rally/serve.py"
click node_bounce "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/bounce/heuristic.py"
click node_winner "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/rally/winner.py"
click node_schema "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/schema/models.py"
click node_fixtures "https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/fixtures.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline,node_video_io,node_court_detector,node_homography,node_court_model,node_players,node_ball_tracker,node_kalman,node_segmentation,node_serve,node_bounce,node_winner toneBlue
class node_api_main,node_api_jobs,node_frontend toneAmber
class node_schema,node_fixtures toneMint
```

## Components

### Engine

Portable analysis engine and point-level inference pipeline

- [Pipeline](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/pipeline.py): engine orchestrator · Coordinates the end-to-end analysis flow and stage injection · `engine/pbengine/pipeline.py`
- [Video probe](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/io/video.py): media IO · Loads and probes input video frames and metadata · `engine/pbengine/io/video.py`
- [Court detect](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/court/detector.py): court finding · Finds court geometry in the source video · `engine/pbengine/court/detector.py`
- [Homography](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/court/homography.py): rectification · Maps camera-space observations into a normalized court frame · `engine/pbengine/court/homography.py`
- [Court model](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/court/court_model.py): court geometry · Represents the canonical pickleball court coordinate system · `engine/pbengine/court/court_model.py`
- [Player track](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/detect/players.py): YOLO + ByteTrack · Detects and tracks players to provide context for point logic · `engine/pbengine/detect/players.py`
- [Ball track](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/ball/tracker.py): ball localization · Locates the ball and produces a rectified trajectory · `engine/pbengine/ball/tracker.py`
- [Kalman](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/ball/kalman.py): trajectory smoothing · Smooths ball motion and stabilizes noisy detections · `engine/pbengine/ball/kalman.py`
- [Rally split](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/rally/segmentation.py): point segmentation · Finds rally boundaries from the tracked events · `engine/pbengine/rally/segmentation.py`
- [Serve logic](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/rally/serve.py): serve heuristic · Identifies serve events and serve-related state transitions · `engine/pbengine/rally/serve.py`
- [Bounce heuristic](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/bounce/heuristic.py): bounce inference · Infers bounce events used by rally outcome logic · `engine/pbengine/bounce/heuristic.py`
- [Winner logic](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/rally/winner.py): point outcome · Classifies rally winners and review flags · `engine/pbengine/rally/winner.py`

### API / UI

Thin FastAPI wrapper and local viewer

- [API app](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/api/app/main.py): FastAPI entrypoint · Starts the web service for uploads and run orchestration · `api/app/main.py`
- [Job state](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/api/app/jobs.py): run coordinator · Tracks analysis jobs and bridges the web layer to engine execution · `api/app/jobs.py`
- [Viewer](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/api/frontend/index.html): single-page UI · Displays uploaded video and per-point results · `api/frontend/index.html`

### Support

Fixtures, schema, and tooling used across runtimes

- [Result schema](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/schema/models.py): JSON contract · Defines the structured output shared by engine, API, and viewer · `engine/pbengine/schema/models.py`
- [Fixtures](https://github.com/thanksmors/pbvision/blob/claude/awesome-albattani-f85twc/engine/pbengine/fixtures.py): synthetic stages · Provides stand-ins for model-backed stages for demo and tests · `engine/pbengine/fixtures.py`

## Connections

- API app → Job state: delegates runs · The web service hands analysis requests to job management
- Job state → Pipeline: launches engine · Jobs execute the analysis pipeline as a separate runtime
- API app → Viewer: serves UI · The API exposes the local viewer for uploaded videos and results
- Pipeline → Video probe: reads video · Pipeline begins by probing the input media
- Pipeline → Court detect: detects court · Finds the playing surface before rectification
- Court detect → Homography: feeds geometry · Court detections are normalized into a stable frame
- Homography → Court model: maps to court · Rectified coordinates are expressed in the canonical court model
- Pipeline → Player track: tracks players · Player detections provide context for later reasoning
- Pipeline → Ball track: tracks ball · Ball localization is the highest-risk perception stage
- Ball track → Kalman: smooths path · Ball trajectories are stabilized before point logic
- Pipeline → Rally split: splits rallies · The pipeline converts trajectories into rally segments
- Rally split → Serve logic: identifies serves · Serve detection helps mark rally starts
- Rally split → Bounce heuristic: infers bounces · Bounce events support downstream winner reasoning
- Rally split → Winner logic: classifies outcome · Rally boundaries and events feed point outcome logic
- Bounce heuristic → Winner logic: supports decisions · Bounce inference is a key input to winner classification
- Pipeline → Result schema: writes result · The engine emits the shared structured JSON contract
- Fixtures → Pipeline: swap stages · Synthetic stand-ins can replace model-backed components
- Fixtures → Rally split: drives demos · Fixtures allow the deterministic rally logic to run without weights
- Fixtures → Winner logic: drives demos · Synthetic inputs exercise point outcome reasoning end to end
- Job state → Result schema: reads result · The web layer consumes the same structured output written by the engine
- Viewer → Result schema: renders result · The viewer presents the shared result model to users