How peng-yt/stt works
Project type and purpose
• This is a local, offline, full-stack speech-to-text web application.
• It provides a browser-based UI for uploading video/audio files, invokes a Flask backend that:
• Calls FFmpeg to preprocess media
• Runs the faster-whisper ASR model (CPU or CUDA)
• Returns transcripts in JSON/SRT/TXT
• It also exposes a POST /api endpoint for programmatic use.
File structure
Top-level:
• start.py – entry-point launching the Flask app
• static/ – JS/CSS (layui), images, tmp storage
• templates/ – index.html (UI view)
• stslib/ – internal library (cfg.py, tool.py)
• models/ – local whisper model weights
• ffmpeg.7z – packaged FFmpeg executables for media I/O
• set.ini – runtime config (devtype=cpu/cuda)
• version.json – app version metadata
• requirements.txt – Python dependencies (Flask, torch, etc.)
Key components
A. Browser (Client UI)
• index.html + layui assets (.js, .css)
• Drag-and-drop or file-choose widget
• Hits HTTP endpoints on Flask server
B. Flask Backend
• start.py sets up routes:
• GET / → renders index.html
• POST /api → handles file + parameters
• stslib.tool.py provides helper functions (transcoding calls, model loading)
• stslib.cfg.py reads set.ini (select CPU vs CUDA)
C. Media Preprocessor (FFmpeg)
• invoked via subprocess for file → WAV chunks
• resides in project root as ffmpeg.exe/ffprobe.exe on Windows or system bin on Linux/Mac
D. ASR Engine (faster-whisper)
• Python module loaded from models/
• Uses PyTorch (CPU or CUDA) for inference
• Outputs timestamped segments
E. Output Formatter
• Converts raw segments into JSON, SRT, or plain text
• Returns HTTP JSON payload
F. Model Storage
• Local filesystem under models/ directory
• User downloads additional weights into models/
G. Configuration & Versioning
• set.ini controls device type
• version.json provides app version
Relationships & Data Flow
1) Browser → Flask (HTTP POST with multipart file + metadata)
2) Flask → stslib.tool → invokes FFmpeg (media preprocessing)
3) Preprocessed audio → faster-whisper ASR model (in-memory via Python API)
4) Transcripts → formatting module → HTTP response
5) Browser renders/transfers result to user UI
Architectural patterns & design principles
• Monolithic web service (single Flask process hosting UI & API)
• MVC-like separation:
• View: templates + static assets
• Controller: Flask route handlers
• Model: faster-whisper weights + stslib modules
• Config-driven (set.ini) for runtime flexibility (CPU vs CUDA)
• Pluggable components: media I/O (FFmpeg) and ASR engine