thanksmors/pbvision

1. Repository shape

- The repo is split into a portable analysis engine under engine/pbengine/ and a thin application layer under api/.

- The engine can run standalone from the command line to produce result.json; the API wraps that engine for uploads and local viewing.

2. End-to-end purpose

- The system analyzes a pickleball match video at the point level.

- The main output is a structured JSON result containing rallies/points, serve and bounce events, winner calls, and confidence / review flags.

3. Top-level data flow

- The pipeline is: video probe → court detection/homography → player tracking → ball tracking → rally segmentation → serve/bounce/winner inference → JSON export.

- The key architectural boundary is between video/ML-heavy perception stages and the deterministic logic stages that operate on rectified coordinates and tracked events.

4. Engine subsystem

- engine/pbengine/pipeline.py is the orchestration entry point.

- It ties together IO, court, player, ball, and rally stages and is designed to run both with real models and with synthetic fixtures.

- Model-backed stages are dependency-injected and can be swapped for test fixtures, which is why the engine is usable without weights for demos and tests.

5. Court geometry subsystem

- court/detector.py, court/homography.py, and court/court_model.py handle court finding and coordinate normalization.

- This subsystem maps camera-space detections into a normalized pickleball court frame, which is crucial because later heuristics assume a stable court coordinate system.

6. Player tracking subsystem

- detect/players.py uses Ultralytics YOLO26 plus ByteTrack.

- Its role is to identify and track players across frames, feeding contextual signals into later rally and serve logic rather than directly producing the final point result.

7. Ball tracking subsystem

- ball/tracker.py and ball/kalman.py handle the highest-risk stage: locating the ball and smoothing its trajectory.

- The README notes this stage uses WASB-SBDT plus jump gating and Kalman smoothing, and the architecture treats it as the most failure-prone component.

- Output from this stage is a rectified ball path used by the rally logic.

8. Rally / point logic subsystem

- rally/segmentation.py, rally/serve.py, and rally/winner.py convert the tracked trajectory into point boundaries and outcomes.

- This layer is mostly heuristic and deterministic: it decides where rallies start/end, how serves are identified, and who won each rally.

- bounce/heuristic.py supports bounce inference, which is a core input to winner classification.

9. Schema and contract

- schema/models.py defines the structured result contract used by both the engine and the API/viewer.

- The JSON schema is the integration boundary: the engine writes it, and the app/front end renders it.

- This contract is what allows the demo fixture pipeline, CLI pipeline, and UI to share one representation.

10. API and viewer

- api/app/main.py exposes the FastAPI service; api/app/jobs.py manages job execution/state.

- The app does not appear to run the ML pipeline inline; instead it spawns the engine as a subprocess and polls a status file, which keeps the web layer thin and isolates the analysis runtime.

- api/frontend/index.html is a single-page viewer for displaying the uploaded video and per-point analysis results.

11. Fixtures and synthetic demo path

- fixtures.py provides scripted stand-ins for the model-backed stages.

- This enables a fully CPU-only demo run and exercises the full pipeline, including segmentation and winner reasoning, without requiring model weights or real footage.

- The tests also lean on this boundary to validate logic independently of ML inference.

12. Testing and support tooling

- engine/tests/ covers the pure-logic pieces: bounce, homography, Kalman, rally, schema, winner, and fixtures.

- scripts/download_weights.sh and scripts/run_local.sh support two modes of use: local app development and model-backed analysis with downloaded weights.

- pyproject.toml likely separates core, API, ML, and dev dependencies, matching the engine/app boundary.

13. Architectural takeaway

- The repo is intentionally organized around a “portable engine + thin UI wrapper” pattern.

- All expensive or uncertain perception work is isolated in the engine, while the API and frontend only coordinate runs and present the resulting point-level analysis.