# vuthanhnhan2003/whisper-project architecture

> Architecture diagram of the GitHub repository vuthanhnhan2003/whisper-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vuthanhnhan2003/whisper-project
- Repository: https://github.com/vuthanhnhan2003/whisper-project

## Overview

1. **What this repo is**
   This is a Python media-transcription service that turns video/audio inputs into subtitle files (`VTT`) and clean text transcripts (`TXT`), with an optional LLM-based post-editing step. The system is centered around a single processing pipeline exposed through an HTTP API.

2. **API boundary**
   `api.py`, `main.py`, and `schemas.py` form the service surface. The API exposes endpoints for:
   - single-video conversion,
   - Google Drive folder batch conversion,
   - job status lookup,
   - listing/downloading generated outputs,
   - health checks.
   
   The request/response models are defined separately, which keeps the API contract distinct from the transcription pipeline.

3. **Job orchestration model**
   Conversion is treated as a job rather than a synchronous request. The API accepts work, stores job state in memory, and background processing updates status until completion. This is important architecturally because the HTTP layer is not the processing engine; it only queues and monitors jobs.

4. **Input ingestion layer**
   The service supports three media sources:
   - direct media URLs such as `.mp4`,
   - streamed sources such as `.m3u8`,
   - Google Drive file/folder links.
   
   Google Drive access depends on `credentials.json` or an API key. This makes Drive a first-class external boundary, not just a convenience feature.

5. **Media acquisition and staging**
   Before transcription, inputs are fetched or captured into temporary local files. For stream sources, `ffmpeg` is part of the runtime path. This means the pipeline depends on local filesystem staging and media tooling rather than streaming transcription directly from the network.

6. **Core transcription engine**
   The transcription stage uses Faster-Whisper. The README shows the pipeline is tuned with parameters for hallucination control, VAD, beam search, repetition penalties, no-speech thresholds, and language selection. Architecturally, this is the central compute-heavy subsystem.

7. **Post-processing and segmentation**
   After transcription, segments are cleaned and rebalanced for subtitle readability. The system targets short subtitle segments, roughly 50–70 characters, so the output is optimized for video display rather than raw ASR chunking.

8. **Optional LLM refinement path**
   A second processing branch can refine transcript text through an external LLM proxy, defaulting to Qwen2.5. The pipeline batches transcript segments, sends them to the proxy, validates JSON-formatted responses, and falls back to Whisper output on timeout or invalid output. This refinement step is an external dependency boundary, not required for the base transcription flow.

9. **Output products and storage**
   The pipeline produces at least two output artifacts:
   - `VTT` for subtitles,
   - `TXT` for downstream RAG ingestion.
   
   Output files are written to local folders and then served back through download/list endpoints. The repo also contains `file_textnodes*` directories and a `rag_textnode_pipeline.py`, indicating a related RAG-oriented transformation path that produces text-node JSON artifacts in addition to plain transcript text.

10. **RAG-oriented variant**
   The presence of `rag_textnode_pipeline.py` and sample/generated `textnodes.json` files suggests there is a specialized pipeline for chunking transcript content into structured text nodes suitable for retrieval systems. This is a separate downstream preparation concern from subtitle generation.

11. **Deployment and runtime**
   The project is containerized with `Dockerfile` and `docker-compose.yml`, and `DEPLOY_DOCKER.md` documents the runtime setup. The container includes `ffmpeg`, and the service is intended to run behind a reverse proxy such as Nginx. There is no database layer visible; persistence is filesystem-based plus in-memory job tracking.

12. **Repo structure implication**
   `pipeline.py` appears to hold the main conversion workflow, while `main.py.backup` and the generated `file_textnodes*` trees indicate iterative development and produced artifacts. The clean architectural picture is: HTTP API at the front, job orchestration in the middle, media/Drive/LLM integrations on the sides, and local file output as the final boundary.

Main entry points: [`pipeline.py`](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py) (Cleanup), [`api.py`](https://github.com/vuthanhnhan2003/whisper-project/blob/main/api.py) (API routes), [`main.py`](https://github.com/vuthanhnhan2003/whisper-project/blob/main/main.py) (Job state), [`rag_textnode_pipeline.py`](https://github.com/vuthanhnhan2003/whisper-project/blob/main/rag_textnode_pipeline.py) (RAG pipeline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API surface"]
  node_main["Main app<br/>entrypoint<br/>[main.py]"]
  node_api_py["API routes<br/>http API<br/>[api.py]"]
  node_schemas["Schemas<br/>models<br/>[schemas.py]"]
end

subgraph group_core["Processing core"]
  node_pipeline["Pipeline<br/>orchestrator<br/>[pipeline.py]"]
  node_job_state(("Job state<br/>in-memory queue<br/>[main.py]"))
  node_whisper{{"Whisper core<br/>ASR engine<br/>[pipeline.py]"}}
  node_postprocess["Cleanup<br/>subtitle shaping<br/>[pipeline.py]"]
end

subgraph group_integrations["External inputs"]
  node_ingest["Input ingest<br/>source adapter<br/>[pipeline.py]"]
  node_media_stage["Media staging<br/>temp files<br/>[pipeline.py]"]
  node_drive[("Google Drive<br/>external source<br/>[pipeline.py]")]
  node_ffmpeg["ffmpeg<br/>media tool<br/>[pipeline.py]"]
  node_llm_refine["LLM refine<br/>optional post-edit<br/>[pipeline.py]"]
end

subgraph group_outputs["Outputs"]
  node_vtt_out["VTT output<br/>subtitle artifact<br/>[pipeline.py]"]
  node_txt_out["TXT output<br/>transcript artifact<br/>[pipeline.py]"]
  node_rag_pipeline["RAG pipeline<br/>text-node pipeline"]
  node_textnodes["Text nodes<br/>JSON artifacts"]
end

subgraph group_deploy["Runtime"]
  node_docker["Container<br/>runtime"]
  node_compose["Compose<br/>deployment<br/>[docker-compose.yml]"]
  node_deploy_doc["Deploy guide<br/>ops docs<br/>[DEPLOY_DOCKER.md]"]
end

node_main -->|"mounts"| node_api_py
node_api_py -->|"validates"| node_schemas
node_api_py -->|"queues"| node_job_state
node_job_state -->|"starts"| node_pipeline
node_pipeline -->|"reads from"| node_ingest
node_ingest -->|"can use"| node_drive
node_ingest -->|"stages"| node_media_stage
node_media_stage -->|"streams via"| node_ffmpeg
node_media_stage -->|"feeds"| node_whisper
node_whisper -->|"emits segments"| node_postprocess
node_postprocess -->|"writes"| node_vtt_out
node_postprocess -->|"writes"| node_txt_out
node_postprocess -.->|"optional branch"| node_llm_refine
node_llm_refine -.->|"fallback to"| node_postprocess
node_txt_out -->|"inputs"| node_rag_pipeline
node_rag_pipeline -->|"generates"| node_textnodes
node_docker -->|"packages"| node_pipeline
node_compose -->|"orchestrates"| node_docker
node_deploy_doc -.->|"documents"| node_compose

click node_main "https://github.com/vuthanhnhan2003/whisper-project/blob/main/main.py"
click node_api_py "https://github.com/vuthanhnhan2003/whisper-project/blob/main/api.py"
click node_schemas "https://github.com/vuthanhnhan2003/whisper-project/blob/main/schemas.py"
click node_pipeline "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_job_state "https://github.com/vuthanhnhan2003/whisper-project/blob/main/main.py"
click node_ingest "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_media_stage "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_drive "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_ffmpeg "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_whisper "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_postprocess "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_llm_refine "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_vtt_out "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_txt_out "https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py"
click node_rag_pipeline "https://github.com/vuthanhnhan2003/whisper-project/blob/main/rag_textnode_pipeline.py"
click node_textnodes "https://github.com/vuthanhnhan2003/whisper-project/blob/main/file_textnodes_v4/Video_2_-_Khái_niệm_triết_học.textnodes.json"
click node_docker "https://github.com/vuthanhnhan2003/whisper-project/tree/main/Dockerfile"
click node_compose "https://github.com/vuthanhnhan2003/whisper-project/blob/main/docker-compose.yml"
click node_deploy_doc "https://github.com/vuthanhnhan2003/whisper-project/blob/main/DEPLOY_DOCKER.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_api_py,node_schemas toneBlue
class node_pipeline,node_job_state,node_whisper,node_postprocess toneAmber
class node_ingest,node_media_stage,node_drive,node_ffmpeg,node_llm_refine toneMint
class node_vtt_out,node_txt_out,node_rag_pipeline,node_textnodes toneRose
class node_docker,node_compose,node_deploy_doc toneIndigo
```

## Components

### API surface

HTTP entrypoints and job contract

- [Main app](https://github.com/vuthanhnhan2003/whisper-project/blob/main/main.py): entrypoint · Service bootstrap and API wiring · `main.py`
- [API routes](https://github.com/vuthanhnhan2003/whisper-project/blob/main/api.py): http API · Conversion, status, health, and downloads · `api.py`
- [Schemas](https://github.com/vuthanhnhan2003/whisper-project/blob/main/schemas.py): models · Request and response contracts · `schemas.py`

### Processing core

Orchestration and transcription pipeline

- [Pipeline](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): orchestrator · End-to-end media conversion workflow · `pipeline.py`
- [Job state](https://github.com/vuthanhnhan2003/whisper-project/blob/main/main.py): in-memory queue · Background job tracking and status updates · `main.py`
- [Whisper core](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): ASR engine · Faster-Whisper transcription with decoding controls · `pipeline.py`
- [Cleanup](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): subtitle shaping · Segment cleanup and subtitle rebalancing · `pipeline.py`

### External inputs

Media, Drive, and optional LLM dependencies

- [Input ingest](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): source adapter · Accepts video URLs, streams, and Drive links · `pipeline.py`
- [Media staging](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): temp files · Fetches media into local filesystem before transcription · `pipeline.py`
- [Google Drive](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): external source · Folder and file access for batch conversion · `pipeline.py`
- [ffmpeg](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): media tool · Captures and prepares streamed media · `pipeline.py`
- [LLM refine](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): optional post-edit · Batched JSON-based transcript refinement with fallback · `pipeline.py`

### Outputs

Persisted artifacts and downstream formats

- [VTT output](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): subtitle artifact · Downloadable subtitle files · `pipeline.py`
- [TXT output](https://github.com/vuthanhnhan2003/whisper-project/blob/main/pipeline.py): transcript artifact · Plain text transcripts for downstream use · `pipeline.py`
- [RAG pipeline](https://github.com/vuthanhnhan2003/whisper-project/blob/main/rag_textnode_pipeline.py): text-node pipeline · Transforms transcripts into structured text nodes · `rag_textnode_pipeline.py`
- [Text nodes](https://github.com/vuthanhnhan2003/whisper-project/blob/main/file_textnodes_v4/Video_2_-_Khái_niệm_triết_học.textnodes.json): JSON artifacts · Structured retrieval-oriented outputs · `file_textnodes_v4/Video_2_-_Khái_niệm_triết_học.textnodes.json`

### Runtime

Containerized execution and deployment

- [Container](https://github.com/vuthanhnhan2003/whisper-project/tree/main/Dockerfile): runtime · Dockerized service runtime with ffmpeg · `Dockerfile`
- [Compose](https://github.com/vuthanhnhan2003/whisper-project/blob/main/docker-compose.yml): deployment · Local multi-service runtime definition · `docker-compose.yml`
- [Deploy guide](https://github.com/vuthanhnhan2003/whisper-project/blob/main/DEPLOY_DOCKER.md): ops docs · Runtime and proxy setup notes · `DEPLOY_DOCKER.md`

## Connections

- Main app → API routes: mounts · Wires routes into the service
- API routes → Schemas: validates · Uses request and response models
- API routes → Job state: queues · Accepts work and tracks background jobs
- Job state → Pipeline: starts · Background execution triggers conversion
- Pipeline → Input ingest: reads from · Selects source type and prepares input
- Input ingest → Google Drive: can use · Supports Google Drive file and folder sources
- Input ingest → Media staging: stages · Downloads or captures media locally
- Media staging → ffmpeg: streams via · Uses ffmpeg for streamed sources
- Media staging → Whisper core: feeds · Provides staged media to ASR
- Whisper core → Cleanup: emits segments · Transcription output is cleaned and rebalanced
- Cleanup → VTT output: writes · Produces subtitle files
- Cleanup → TXT output: writes · Produces plain transcript text
- Cleanup → LLM refine: optional branch · Can route segments through external LLM refinement
- LLM refine → Cleanup: fallback to · Invalid or timed-out refinement returns Whisper text
- TXT output → RAG pipeline: inputs · Downstream RAG transformation consumes transcript text
- RAG pipeline → Text nodes: generates · Creates structured text-node JSON artifacts
- Container → Pipeline: packages · Runtime includes transcription toolchain
- Compose → Container: orchestrates · Defines containerized execution
- Deploy guide → Compose: documents · Explains deployment and proxy setup
