# robertmpevec/clip_genius architecture

> Architecture diagram of the GitHub repository robertmpevec/clip_genius, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/robertmpevec/clip_genius
- Repository: https://github.com/robertmpevec/clip_genius

## Overview

##### Project type and purpose
- This is a full-stack, video-processing and analytics application called “Scoreboard Analyzer AI.”
- Purpose: ingest raw sports video, detect and track scoreboard changes via OCR, analyze commentary with NLP to rank exciting moments, generate short highlight clips, and serve them through a web UI.

##### File structure
- Top-level folders:
- backend/ – Python modules for processing, services, models, database and main API entrypoint (main.py)
- frontend/ – web UI code: app.js, static assets and a Python “clip_genius_application.py” (likely orchestration or notebook)
- deployment/ – Dockerfile and Kubernetes manifest
- database files under backend/database (SQL schema, db_handler.py)
- tests/ – unit and integration tests
- Architectural cues:
- backend/Processing – image pre-processing, OCR wrapper, parallelization
- backend/services – separate scripts for clip merging, vector DB (FAISS), search/classification
- backend/models – model utility functions (likely wrapping NeMo NLP models)

##### Pull insights from README
- Core tech: OpenCV, PyTesseract OCR, NVIDIA NeMo NLP, FFmpeg, FAISS vector DB
- Workflow: video → OCR scoreboard detection → NLP commentary analysis → clip ranking → clip generation
- Targets real-time score tracking and automated highlight creation

##### Main components
a. Video Ingestion
- Component: “Video Input Service”
- In: raw video files (clips/ directory or streaming endpoint)
b. Pre-Processing & OCR
- backend/Processing/PreProcessing.py & ImageProcessingFunctions.py
- Calls: OpenCV → frame extraction, normalization → PyTesseract OCR → scoreboard text
c. NLP Commentary Analysis
- backend/models/model_utils.py + NeMo models
- Input: audio transcript or text feed → output: excitement score
d. Clip Generation Service
- backend/services/process_video.py + FFmpeg calls
- Merges top-ranked segments (see merged_video.mp4)
e. Data Stores
- Relational DB (backend/database/database-schema.sql + db_handler.py) → stores timestamped scores
- FAISS Vector DB (backend/services/vector_database.py + highlight_vectors.faiss) → nearest-neighbor search on NLP embeddings
f. API Layer / Orchestration
- backend/main.py → REST endpoints for: submit video, get scoreboard timeline, fetch highlights
g. Frontend UI
- frontend/app.js + static/PresentationFormatter.js, ReportsFormatter.js
- Interacts via AJAX/REST with backend/main.py
h. Deployment & Infrastructure
- Docker container per service (monolithic backend + frontend or separate containers)
- Kubernetes manifest (deployment/k8s.yaml) for scaling, service discovery

##### Relationships and interactions (data flow)
1. User uploads or points to video → sent to Video Input Service
2. Video frames → Pre-Processing → OCR → Scoreboard DB writes
3. Transcript feed → NLP Service → embeddings stored in FAISS
4. Backend scheduler or user request → Clip Generation Service pulls high-excitement segments → produces merged highlights
5. Frontend UI fetches scoreboard data and highlight URLs via REST → renders video player and stats

##### Architectural patterns & design principles
- Pipeline pattern: clearly separated sequential stages (Pre-Processing → OCR → NLP → Clip Gen)
- Modular/service-oriented backend: distinct Python modules under services/ for clipping, vector search, classification
- Data persistence separation: relational vs. vector DB for low-latency similarity search
- Containerization + orchestration for reproducibility and scalability

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User and Frontend Layer
    subgraph "User & Frontend Layer"
        Browser["User Browser"]:::ui
        Frontend["Frontend UI (frontend/app.js)"]:::ui
    end

    %% API Layer
    subgraph "API Layer"
        API["API Layer (backend/main.py)"]:::compute
    end

    %% Processing Pipeline
    subgraph "Processing Pipeline"
        VideoInput["Video Input Service"]:::compute
        PreProc["Pre-Processing & OCR"]:::compute
        NLP["NLP Commentary Analysis"]:::compute
        ClipGen["Clip Generation Service"]:::compute
    end

    %% Data Stores
    subgraph "Data Stores"
        SQLDB["SQL Database"]:::db
        FAISS["FAISS Vector DB"]:::db
    end

    %% External Libraries
    subgraph "External Libraries"
        OpenCV["OpenCV"]:::external
        Tesseract["PyTesseract OCR"]:::external
        NeMo["NVIDIA NeMo"]:::external
        FFmpeg["FFmpeg"]:::external
    end

    %% Deployment & Infrastructure
    subgraph "Deployment & Infrastructure"
        Docker["Docker Container"]:::infra
        K8s["Kubernetes (deployment/k8s.yaml)"]:::infra
    end

    %% Connections
    Browser --> Frontend
    Frontend --> API
    API --> VideoInput
    VideoInput --> PreProc
    PreProc --> SQLDB
    PreProc --> OpenCV
    PreProc --> Tesseract
    API --> NLP
    NLP --> FAISS
    NLP --> NeMo
    API --> ClipGen
    ClipGen --> FFmpeg
    ClipGen --> SQLDB
    ClipGen --> FAISS
    Frontend -->|fetch highlights| API

    API --> Docker
    Docker --> K8s

    %% Click Events
    click VideoInput "https://github.com/robertmpevec/clip_genius/tree/main/clips/"
    click PreProc "https://github.com/robertmpevec/clip_genius/blob/main/backend/Processing/PreProcessing.py"
    click PreProc "https://github.com/robertmpevec/clip_genius/blob/main/backend/Processing/ImageProcessingFunctions.py"
    click PreProc "https://github.com/robertmpevec/clip_genius/blob/main/backend/Processing/MultiProcessing.py"
    click PreProc "https://github.com/robertmpevec/clip_genius/blob/main/backend/Processing/Processor.py"
    click NLP "https://github.com/robertmpevec/clip_genius/blob/main/backend/models/model_utils.py"
    click ClipGen "https://github.com/robertmpevec/clip_genius/blob/main/backend/services/process_video.py"
    click SQLDB "https://github.com/robertmpevec/clip_genius/blob/main/backend/database/database-scheama.sql"
    click SQLDB "https://github.com/robertmpevec/clip_genius/blob/main/backend/database/db_handler.py"
    click FAISS "https://github.com/robertmpevec/clip_genius/blob/main/backend/services/vector_database.py"
    click FAISS "https://github.com/robertmpevec/clip_genius/blob/main/backend/services/highlight_vectors.faiss"
    click API "https://github.com/robertmpevec/clip_genius/blob/main/backend/main.py"
    click Frontend "https://github.com/robertmpevec/clip_genius/blob/main/frontend/app.js"
    click Frontend "https://github.com/robertmpevec/clip_genius/blob/main/frontend/static/PresentationFormatter.js"
    click Frontend "https://github.com/robertmpevec/clip_genius/blob/main/frontend/static/ReportsFormatter.js"
    click Frontend "https://github.com/robertmpevec/clip_genius/blob/main/frontend/clip_genius_application.py"
    click Docker "https://github.com/robertmpevec/clip_genius/tree/main/deployment/Dockerfile"
    click K8s "https://github.com/robertmpevec/clip_genius/blob/main/deployment/k8s.yaml"

    %% Styles
    classDef compute fill:#c2f0c2,stroke:#2d572c
    classDef db fill:#c2d6f0,stroke:#1d3b73
    classDef ui fill:#f0c2c2,stroke:#721d1d
    classDef external fill:#d3d3d3,stroke:#888
    classDef infra fill:#e2e2e2,stroke:#555
```
