How vuthanhnhan2003/whisper-project works
1. What this repo is
This is a Python media-transcription service that turns video/audio inputs into subtitle files (VTT) and clean text transcripts (TXT), with an optional LLM-based post-editing step. The system is centered around a single processing pipeline exposed through an HTTP API.
2. API boundary
api.py, main.py, and schemas.py form the service surface. The API exposes endpoints for:
- single-video conversion,
- Google Drive folder batch conversion,
- job status lookup,
- listing/downloading generated outputs,
- health checks.
The request/response models are defined separately, which keeps the API contract distinct from the transcription pipeline.
3. Job orchestration model
Conversion is treated as a job rather than a synchronous request. The API accepts work, stores job state in memory, and background processing updates status until completion. This is important architecturally because the HTTP layer is not the processing engine; it only queues and monitors jobs.
4. Input ingestion layer
The service supports three media sources:
- direct media URLs such as .mp4,
- streamed sources such as .m3u8,
- Google Drive file/folder links.
Google Drive access depends on credentials.json or an API key. This makes Drive a first-class external boundary, not just a convenience feature.
5. Media acquisition and staging
Before transcription, inputs are fetched or captured into temporary local files. For stream sources, ffmpeg is part of the runtime path. This means the pipeline depends on local filesystem staging and media tooling rather than streaming transcription directly from the network.
6. Core transcription engine
The transcription stage uses Faster-Whisper. The README shows the pipeline is tuned with parameters for hallucination control, VAD, beam search, repetition penalties, no-speech thresholds, and language selection. Architecturally, this is the central compute-heavy subsystem.
7. Post-processing and segmentation
After transcription, segments are cleaned and rebalanced for subtitle readability. The system targets short subtitle segments, roughly 50–70 characters, so the output is optimized for video display rather than raw ASR chunking.
8. Optional LLM refinement path
A second processing branch can refine transcript text through an external LLM proxy, defaulting to Qwen2.5. The pipeline batches transcript segments, sends them to the proxy, validates JSON-formatted responses, and falls back to Whisper output on timeout or invalid output. This refinement step is an external dependency boundary, not required for the base transcription flow.
9. Output products and storage
The pipeline produces at least two output artifacts:
- VTT for subtitles,
- TXT for downstream RAG ingestion.
Output files are written to local folders and then served back through download/list endpoints. The repo also contains file_textnodes* directories and a rag_textnode_pipeline.py, indicating a related RAG-oriented transformation path that produces text-node JSON artifacts in addition to plain transcript text.
10. RAG-oriented variant
The presence of rag_textnode_pipeline.py and sample/generated textnodes.json files suggests there is a specialized pipeline for chunking transcript content into structured text nodes suitable for retrieval systems. This is a separate downstream preparation concern from subtitle generation.
11. Deployment and runtime
The project is containerized with Dockerfile and docker-compose.yml, and DEPLOY_DOCKER.md documents the runtime setup. The container includes ffmpeg, and the service is intended to run behind a reverse proxy such as Nginx. There is no database layer visible; persistence is filesystem-based plus in-memory job tracking.
12. Repo structure implication
pipeline.py appears to hold the main conversion workflow, while main.py.backup and the generated file_textnodes* trees indicate iterative development and produced artifacts. The clean architectural picture is: HTTP API at the front, job orchestration in the middle, media/Drive/LLM integrations on the sides, and local file output as the final boundary.
Main entry points: pipeline.py Cleanup · api.py API routes · main.py Job state · rag_textnode_pipeline.py RAG pipeline
Components 19
API surface
HTTP entrypoints and job contract
- Main app · Service bootstrap and API wiring
main.py - API routes · Conversion, status, health, and downloads
api.py - Schemas · Request and response contracts
schemas.py
Processing core
Orchestration and transcription pipeline
- Pipeline · End-to-end media conversion workflow
pipeline.py - Job state · Background job tracking and status updates
main.py - Whisper core · Faster-Whisper transcription with decoding controls
pipeline.py - Cleanup · Segment cleanup and subtitle rebalancing
pipeline.py
External inputs
Media, Drive, and optional LLM dependencies
- Input ingest · Accepts video URLs, streams, and Drive links
pipeline.py - Media staging · Fetches media into local filesystem before transcription
pipeline.py - Google Drive · Folder and file access for batch conversion
pipeline.py - ffmpeg · Captures and prepares streamed media
pipeline.py - LLM refine · Batched JSON-based transcript refinement with fallback
pipeline.py
Outputs
Persisted artifacts and downstream formats
- VTT output · Downloadable subtitle files
pipeline.py - TXT output · Plain text transcripts for downstream use
pipeline.py - RAG pipeline · Transforms transcripts into structured text nodes
rag_textnode_pipeline.py - Text nodes · Structured retrieval-oriented outputs
file_textnodes_v4/Video_2_-_Khái_niệm_triết_học.textnodes.json
Runtime
Containerized execution and deployment
- Container · Dockerized service runtime with ffmpeg
Dockerfile - Compose · Local multi-service runtime definition
docker-compose.yml - Deploy guide · Runtime and proxy setup notes
DEPLOY_DOCKER.md
Connections 19
- Main app → API routes: mounts · Wires routes into the service
- API routes → Schemas: validates · Uses request and response models
- API routes → Job state: queues · Accepts work and tracks background jobs
- Job state → Pipeline: starts · Background execution triggers conversion
- Pipeline → Input ingest: reads from · Selects source type and prepares input
- Input ingest → Google Drive: can use · Supports Google Drive file and folder sources
- Input ingest → Media staging: stages · Downloads or captures media locally
- Media staging → ffmpeg: streams via · Uses ffmpeg for streamed sources
- Media staging → Whisper core: feeds · Provides staged media to ASR
- Whisper core → Cleanup: emits segments · Transcription output is cleaned and rebalanced
- Cleanup → VTT output: writes · Produces subtitle files
- Cleanup → TXT output: writes · Produces plain transcript text
- Cleanup → LLM refine: optional branch · Can route segments through external LLM refinement
- LLM refine → Cleanup: fallback to · Invalid or timed-out refinement returns Whisper text
- TXT output → RAG pipeline: inputs · Downstream RAG transformation consumes transcript text
- RAG pipeline → Text nodes: generates · Creates structured text-node JSON artifacts
- Container → Pipeline: packages · Runtime includes transcription toolchain
- Compose → Container: orchestrates · Defines containerized execution
- Deploy guide → Compose: documents · Explains deployment and proxy setup