How yaofanguk/video-subtitle-extractor works
Project Type and Purpose
• This project, known as Video-subtitle-extractor (VSE), is a full-fledged desktop application (with both CLI and GUI entry points) designed to extract hard subtitles from videos and output them as srt (or txt) files.
• Its functionality covers video keyframe extraction, text detection, OCR-based text recognition, text filtering (e.g., watermarks or unwanted text), and subtitle timing formation.
• It supports batch processing and multiple languages (up to 87 languages) with options that trade off speed and accuracy (fast, automatic, and precise detection).
File Structure
• The top-level structure includes configuration files (.condarc, requirements.txt, etc.), a README, and multiple directories:
• The “backend” directory holds the core processing code including configuration (config.py, configs/typoMap.json), model directories (models/V2, V3, V4), and an OCR sub-module “ppocr” whose extensive structure covers data processing, losses, backbones, heads, postprocessing, utilities, etc.
• The “backend/interface” directory holds language-specific files (e.g., ch.ini, en.ini), suggesting localization support.
• The “tools” directory contains scripts for evaluation, inference, export, and training.
• The “subfinder” directory indicates an additional functionality for subtitle extraction in videos across different platforms.
• The “gui.py” at root indicates the graphical user interface module.
• The “test” folder holds sample video files for testing.
• The file tree also includes design assets (design folder) and notebooks for Google Colab, indicating support for both local usage and an online GPU-enabled environment.
Main Components and Their Interactions
● User Interface Layer:
• GUI (gui.py) for interactive usage.
• CLI (backend/main.py) for command-line invocation.
● Video Processing Module:
• Video loader and keyframe extraction. This can be seen inside the tools and backend, where input videos are processed.
● Pre-Processing Module:
• Image enhancement, filtering, and preparation of frames before sending them to OCR.
● OCR/Detection Module (PP-OCR):
• Includes deep learning modules for text detection and recognition. This is housed in backend/ppocr with subdirectories for “data”, “losses”, “modeling”, “optimizer”, “postprocess”, and “utils”. It leverages PaddlePaddle for GPU acceleration.
● Language and Configuration Management:
• The “backend/interface” directory and config files (like config.py and typoMap.json) handle language-specific settings and custom text filtering (e.g., replacing or removing watermark text).
● Post-Processing Module:
• This layer processes the OCR outputs, removes duplicates, applies text filtering, and generates srt/txt files.
● External Tools and Integration:
• Tools for exporting models, running evaluation, and inference (under backend/tools/infer) should be shown as supporting utilities.
• Support for GPU acceleration through PaddlePaddle.
● Data Flow:
• The configuration files and dictionaries (from backend/ppocr/utils/dict) become part of the language processing and model configuration components.
Architectural Patterns and Design Principles
• The system has a modular design:
• Separation of concerns between the frontend (GUI/CLI), core OCR processing (PP-OCR and inference), configuration/language handling, and post-processing.
• It uses clear layering: input processing, inference (model based on deep learning components), and postprocessing.
• There is a separation between model versions (V2, V3, V4) that cater to different trade-offs between speed and accuracy, a design choice visible in the file tree.
• Components should be clearly separated and labeled. For instance:
• Suggest adding notes on error handling and configuration linkage (e.g., path restrictions like “no Chinese/space in path”).