# cicada000/vv architecture

> Architecture diagram of the GitHub repository cicada000/vv, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cicada000/vv
- Repository: https://github.com/cicada000/vv

## Overview

##### Main Components
- Data Processing & Preprocessing Pipeline:
- This module is responsible for processing the raw video assets and generating the searchable data.
- It includes the DataProcess folder scripts (compress_subtitle.py, create_db.py, import_subtitles.py, merge_screenshot.py, search_subtitles.py, and main.py) that generate JSON files with subtitle data.
- It also encompasses the subtitle extraction scripts (CutSubtitle.py and CutSubtitle_paddleocr.py) that use different OCR libraries (ddddocr and PaddleOCR).

- Face Recognition Engine:
- There are two implementations here: one using dlib in FaceRec.py and another using insightface in FaceRec_insightface.py.
- The modules generate_features.py and generate_features_insightface.py are used for precomputing face features, outputting files such as face_features.npz.
- This engine is integral during preprocessing to tag video frames with face features.

- Search Engine & Matching Module:
- Found in the “search” folder, this contains both “cloud” (with TypeScript, Node, and worker scripts) and “local” (Python-based) implementations.
- These modules are responsible for processing user queries and performing fuzzy text matching against the subtitle JSON files.

- API Backend:
- Located in the “api” folder, these scripts (bot.py, index.py, and the subfolder “search” that contains a Rust-based search engine) provide HTTP endpoints.
- The API layer exposes search functionality (e.g., handling query parameters such as query keywords, minimum text match ratio, and face similarity thresholds) and interacts with both the search engine and the data generated earlier.
- There is also an “api/subtitle_search_api” that likely wraps these functionalities into RESTful services.

- Web Frontend:
- The “Web” folder contains the client-side code (HTML, CSS, JS) that users interact with at https://vv.cicada000.work.
- It includes files such as index.html, db_search.js, and script.js along with configuration files (e.g., mapping.json) to display search results.

##### Relationships and Data Flow
- The data processing pipeline (including subtitle extraction and face recognition) is run either via main.py or as part of a preprocessing batch. Its output (e.g., JSON files in the “subtitle” directory) is stored in a database (or file-based storage) created via create_db.py.
- The face recognition component (using either dlib or insightface) annotates video frames to assist with verifying the “VV” faces during query matching.
- The search engine reads the subtitle data and leverages vector/text similarity algorithms (using both local Python scripts and Node/TS code in the cloud folder) for fuzzy matching.
- The API backend sits on top of the search engine and data store and exposes endpoints that the Web frontend calls. An example API call is provided in the README.
- The Web frontend (client user interface) sends search requests to the backend API, receives test results, and displays them to the user.

##### Architectural Patterns & Design Principles
- Layered Architecture: The project separates concerns among data ingestion/preprocessing, data retrieval/search, and client interfacing.
- Modular Design: Different recognition methods (dlib vs. insightface) and search approaches (local vs. cloud) are neatly separated into distinct modules.
- Extensibility: The design supports adding new video sources or recognition methods (as mentioned in the README) and can be extended or modified independently.

##### Technologies/Frameworks/Libraries
- Python is used for most processing, recognition (with ddddocr, PaddleOCR, dlib, insightface), and orchestration.
- Rust (in the api/search subdirectory) suggests use of performance-oriented search algorithms.
- Web technologies (HTML, CSS, JavaScript, TypeScript) power the frontend and parts of the “cloud” search.
- Deployment/configuration is facilitated by vercel.json and other configuration files.

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% Preprocessing Pipeline Subgraph
    subgraph "Preprocessing Pipeline"
        DP["DataProcess Pipeline"]:::preProcessing
        SE1["Subtitle Extraction (ddddocr)"]:::preProcessing
        SE2["Subtitle Extraction (PaddleOCR)"]:::preProcessing
        subgraph "Face Recognition Engine"
            FR1["FaceRec (dlib)"]:::recognition
            FR2["FaceRec (insightface)"]:::recognition
            GF1["Generate Features (dlib)"]:::recognition
            GF2["Generate Features (insightface)"]:::recognition
            FF1["Face Features (dlib)"]:::recognition
            FF2["Face Features (insightface)"]:::recognition
        end
    end

    %% Data Storage Node
    DS["Data Aggregation & DB Creation"]:::storage

    %% Search Engine Subgraph
    subgraph "Search Engine"
        CS["Cloud Search"]:::search
        LS["Local Search"]:::search
    end

    %% API Backend Subgraph
    subgraph "API Backend"
        API["Python API"]:::api
        RS["Rust Search Engine"]:::api
        SA["Subtitle Search API"]:::api
    end

    %% Web Frontend Node
    WF["Web UI"]:::frontend

    %% Arrows from Preprocessing Pipeline to Data Storage
    DP -->|"JSONData"| DS
    SE1 -->|"OCRData"| DS
    SE2 -->|"OCRData"| DS
    FR1 -->|"faceData"| FF1
    GF1 -->|"features"| FF1
    FR2 -->|"faceData"| FF2
    GF2 -->|"features"| FF2
    FF1 -->|"annotates"| DS
    FF2 -->|"annotates"| DS

    %% Arrows from Data Storage to Search Engine
    DS -->|"QueryData"| CS
    DS -->|"QueryData"| LS

    %% Arrows from Search Engine to API Backend
    CS -->|"SearchQuery"| API
    LS -->|"SearchQuery"| API

    %% Arrows within API Backend
    API -->|"ExecRustSearch"| RS
    API -->|"SubtitleAPICall"| SA

    %% Arrows between API Backend and Web Frontend
    WF -- "RESTRequest" --> API
    API -- "HTTPResponse" --> WF

    %% Click Events for Preprocessing Pipeline
    click DP "https://github.com/cicada000/vv/tree/main/DataProcess"
    click SE1 "https://github.com/cicada000/vv/blob/main/CutSubtitle.py"
    click SE2 "https://github.com/cicada000/vv/blob/main/CutSubtitle_paddleocr.py"

    %% Click Events for Face Recognition Engine
    click FR1 "https://github.com/cicada000/vv/blob/main/FaceRec.py"
    click FR2 "https://github.com/cicada000/vv/blob/main/FaceRec_insightface.py"
    click GF1 "https://github.com/cicada000/vv/blob/main/generate_features.py"
    click GF2 "https://github.com/cicada000/vv/blob/main/generate_features_insightface.py"
    click FF1 "https://github.com/cicada000/vv/blob/main/face_features.npz"
    click FF2 "https://github.com/cicada000/vv/blob/main/face_features_insightface.npz"

    %% Click Events for Search Engine
    click CS "https://github.com/cicada000/vv/tree/main/search/cloud"
    click LS "https://github.com/cicada000/vv/tree/main/search/local"

    %% Click Events for API Backend
    click API "https://github.com/cicada000/vv/blob/main/api/index.py"
    click RS "https://github.com/cicada000/vv/tree/main/api/search"
    click SA "https://github.com/cicada000/vv/tree/main/api/subtitle_search_api"

    %% Click Event for Web Frontend
    click WF "https://github.com/cicada000/vv/tree/main/Web"

    %% Styles
    classDef preProcessing fill:#f9c,stroke:#333,stroke-width:2px;
    classDef recognition fill:#cfc,stroke:#333,stroke-width:2px;
    classDef storage fill:#afa,stroke:#333,stroke-width:2px;
    classDef search fill:#ff9,stroke:#333,stroke-width:2px;
    classDef api fill:#9cf,stroke:#333,stroke-width:2px;
    classDef frontend fill:#fc9,stroke:#333,stroke-width:2px;
```
