# yaofanguk/video-subtitle-extractor architecture

> Architecture diagram of the GitHub repository yaofanguk/video-subtitle-extractor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yaofanguk/video-subtitle-extractor
- Repository: https://github.com/yaofanguk/video-subtitle-extractor

## Overview

##### Project Type and Purpose
- This project, known as Video-subtitle-extractor (VSE), is a full-fledged desktop application (with both CLI and GUI entry points) designed to extract hard subtitles from videos and output them as srt (or txt) files.
- Its functionality covers video keyframe extraction, text detection, OCR-based text recognition, text filtering (e.g., watermarks or unwanted text), and subtitle timing formation.
- It supports batch processing and multiple languages (up to 87 languages) with options that trade off speed and accuracy (fast, automatic, and precise detection).

##### File Structure
- The top-level structure includes configuration files (.condarc, requirements.txt, etc.), a README, and multiple directories:
- The “backend” directory holds the core processing code including configuration (config.py, configs/typoMap.json), model directories (models/V2, V3, V4), and an OCR sub-module “ppocr” whose extensive structure covers data processing, losses, backbones, heads, postprocessing, utilities, etc.
- The “backend/interface” directory holds language-specific files (e.g., ch.ini, en.ini), suggesting localization support.
- The “tools” directory contains scripts for evaluation, inference, export, and training.
- The “subfinder” directory indicates an additional functionality for subtitle extraction in videos across different platforms.
- The “gui.py” at root indicates the graphical user interface module.
- The “test” folder holds sample video files for testing.
- The file tree also includes design assets (design folder) and notebooks for Google Colab, indicating support for both local usage and an online GPU-enabled environment.

##### Main Components and Their Interactions
● User Interface Layer:
- GUI (gui.py) for interactive usage.
- CLI (backend/main.py) for command-line invocation.
● Video Processing Module:
- Video loader and keyframe extraction. This can be seen inside the tools and backend, where input videos are processed.
● Pre-Processing Module:
- Image enhancement, filtering, and preparation of frames before sending them to OCR.
● OCR/Detection Module (PP-OCR):
- Includes deep learning modules for text detection and recognition. This is housed in backend/ppocr with subdirectories for “data”, “losses”, “modeling”, “optimizer”, “postprocess”, and “utils”. It leverages PaddlePaddle for GPU acceleration.
● Language and Configuration Management:
- The “backend/interface” directory and config files (like config.py and typoMap.json) handle language-specific settings and custom text filtering (e.g., replacing or removing watermark text).
● Post-Processing Module:
- This layer processes the OCR outputs, removes duplicates, applies text filtering, and generates srt/txt files.
● External Tools and Integration:
- Tools for exporting models, running evaluation, and inference (under backend/tools/infer) should be shown as supporting utilities.
- Support for GPU acceleration through PaddlePaddle.
● Data Flow:
- The configuration files and dictionaries (from backend/ppocr/utils/dict) become part of the language processing and model configuration components.

##### Architectural Patterns and Design Principles
- The system has a modular design:
- Separation of concerns between the frontend (GUI/CLI), core OCR processing (PP-OCR and inference), configuration/language handling, and post-processing.
- It uses clear layering: input processing, inference (model based on deep learning components), and postprocessing.
- There is a separation between model versions (V2, V3, V4) that cater to different trade-offs between speed and accuracy, a design choice visible in the file tree.

- Components should be clearly separated and labeled. For instance:
- Suggest adding notes on error handling and configuration linkage (e.g., path restrictions like “no Chinese/space in path”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface Layer
    subgraph "User Interface Layer"
        UI1["GUI (Graphical User Interface)"]:::ui
        UI2["CLI (Command Line Interface)"]:::ui
    end

    %% Video Processing Module
    subgraph "Video Processing Module"
        VP1["Video Loader & Keyframe Extraction"]:::processing
        VP2["Video Samples"]:::processing
    end

    %% Pre-Processing Module
    subgraph "Pre-Processing Module"
        PP["Frame Preparation & Filtering"]:::processing
    end

    %% OCR/Detection Module (PP-OCR Core Engine)
    subgraph "OCR/Detection Module (PP-OCR Core Engine)"
        OCR["PP-OCR Engine"]:::ocr
        M2["Model V2"]:::ocr
        M3["Model V3"]:::ocr
        M4["Model V4"]:::ocr
    end

    %% Language & Configuration Management
    subgraph "Language & Configuration Management"
        CFG1["config.py"]:::config
        CFG2["typoMap.json"]:::config
        CFG3["Language Files"]:::config
    end

    %% Post-Processing Module
    subgraph "Post-Processing Module"
        PS1["Subtitle Generation (Tools)"]:::post
        PS2["PostProcessing Scripts"]:::post
        OUT["Subtitle Files (SRT/TXT)"]:::post
    end

    %% External Tools & Integration
    subgraph "External Tools & Integration"
        EX1["Inference Scripts"]:::extern
        EX2["Export Model"]:::extern
        EX3["Evaluation"]:::extern
    end

    %% Additional Subtitle Extraction
    subgraph "Additional Subtitle Extraction"
        SF["Subfinder (Multi-platform)"]:::extern
    end

    %% Data Flow Connections
    UI1 -->|"triggers"| VP1
    UI2 -->|"triggers"| VP1
    VP2 -->|"provides input"| VP1
    VP1 -->|"extracts frames"| PP
    PP -->|"prepares frames"| OCR
    OCR -->|"selects model"| M2
    OCR --> M3
    OCR --> M4
    OCR -->|"outputs text"| PS1
    PS1 -->|"refines output"| PS2
    PS2 -->|"generates"| OUT
    CFG1 ---|"configures"| OCR
    CFG2 --- OCR
    CFG3 --- OCR
    CFG1 ---|"guides filtering"| PS2
    CFG2 --- PS2
    CFG3 --- PS2
    EX1 ---|"supports"| OCR
    EX2 --- OCR
    EX3 --- OCR
    VP1 ---|"optional extraction"| SF

    %% Class Definitions for Styling
    classDef ui fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef processing fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef ocr fill:#A9DFBF,stroke:#196F3D,stroke-width:2px;
    classDef config fill:#D6EAF8,stroke:#2471A3,stroke-width:2px;
    classDef post fill:#F5B7B1,stroke:#943126,stroke-width:2px;
    classDef extern fill:#D5DBDB,stroke:#5D6D7E,stroke-width:2px;

    %% Click Events
    click UI1 "https://github.com/yaofanguk/video-subtitle-extractor/blob/main//gui.py"
    click UI2 "https://github.com/yaofanguk/video-subtitle-extractor/blob/main//backend/main.py"
    click VP1 "https://github.com/yaofanguk/video-subtitle-extractor/blob/main//backend/tools/subtitle_ocr.py"
    click PP "https://github.com/yaofanguk/video-subtitle-extractor/blob/main//backend/tools/ocr.py"
    click OCR "https://github.com/yaofanguk/video-subtitle-extractor/tree/main//backend/ppocr"
    click M2 "https://github.com/yaofanguk/video-subtitle-extractor/tree/main//backend/models/V2"
    click M3 "https://github.com/yaofanguk/video-subtitle-extractor/tree/main//backend/models/V3"
    click M4 "https://github.com/yaofanguk/video-subtitle-extractor/tree/main//backend/models/V4"
    click CFG1 "https://github.com/yaofanguk/video-subtitle-extractor/blob/main//backend/config.py"
    click CFG2 "https://github.com/yaofanguk/video-subtitle-extractor/blob/main//backend/configs/typoMap.json"
    click CFG3 "https://github.com/yaofanguk/video-subtitle-extractor/tree/main//backend/interface"
    click PS1 "https://github.com/yaofanguk/video-subtitle-extractor/blob/main//backend/tools/subtitle_ocr.py"
    click EX1 "https://github.com/yaofanguk/video-subtitle-extractor/tree/main//backend/tools/infer"
    click EX2 "https://github.com/yaofanguk/video-subtitle-extractor/blob/main//backend/tools/export_model.py"
    click EX3 "https://github.com/yaofanguk/video-subtitle-extractor/blob/main//backend/tools/eval.py"
    click SF "https://github.com/yaofanguk/video-subtitle-extractor/tree/main//backend/subfinder"
```
