# dimastatz/whisper-flow architecture

> Architecture diagram of the GitHub repository dimastatz/whisper-flow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dimastatz/whisper-flow
- Repository: https://github.com/dimastatz/whisper-flow

## Overview

1. **Repository shape**
   - This is a Python package plus a small service layer for real-time speech-to-text.
   - The core package is `whisperflow/`; everything else is supporting infrastructure: tests, docs, Docker, and developer tooling.

2. **Primary runtime boundary**
   - The main runtime is a **FastAPI-based transcription service** in `whisperflow/fast_server.py`.
   - It exposes:
     - a **WebSocket streaming endpoint** for live audio,
     - a **health check** endpoint,
     - and an **HTTP batch endpoint** for transcribing PCM chunks.
   - This server is the integration boundary for external clients.

3. **Streaming transcription pipeline**
   - The central flow is: **audio chunks enter → buffering/session logic groups them → Whisper transcription runs → partial/final text is sent back**.
   - `whisperflow/streaming.py` appears to own the session orchestration, including incremental updates and lifecycle management.
   - The README shows a callback-driven design: one callback to transcribe chunks, another to send results back to the client.

4. **Model/transcription subsystem**
   - `whisperflow/transcriber.py` is the model-facing layer.
   - It loads the Whisper model and performs transcription over PCM audio chunks.
   - The packaged model file `whisperflow/models/tiny.en.pt` indicates the repository ships with a default lightweight model for local use and demos.

5. **Audio input boundary**
   - `whisperflow/audio/microphone.py` is the local capture path for live microphone audio.
   - The README and example script suggest it is used for interactive/demo scenarios rather than being required for server operation.
   - This creates a second ingestion path distinct from network streaming: local microphone → PCM chunks → same transcription pipeline.

6. **Data format contract**
   - The system expects **PCM audio** with a fixed contract: **16 kHz, mono, 16-bit signed integer**.
   - That format boundary is important because it simplifies the transcriber and makes the service predictable for both WebSocket clients and benchmark inputs.

7. **Response semantics**
   - Output is not just a single transcript; the system emits **partial results over time** and then a final segment completion event.
   - Each result includes timing/partiality metadata, which is what enables low-latency “live” UX rather than batch-only transcription.

8. **Benchmarking and evaluation**
   - `tests/benchmark/` and the README’s LibriSpeech-based benchmark establish a performance-focused subsystem.
   - The main metrics are **latency** between partial updates and **word error rate (WER)**.
   - This is architecturally important because it validates the streaming segmentation strategy, not just raw transcription accuracy.

9. **Test coverage of the core boundaries**
   - The tests are organized around the main modules: audio handling, streaming session behavior, transcriber behavior, chat-room/session behavior, and benchmark execution.
   - This suggests the repo’s critical logic is concentrated in a few boundaries and the tests are aligned to them rather than to many small utilities.

10. **Packaging and deployment**
   - The project is packaged for installation via `setup.py` and dependency pinning in `requirements.txt`.
   - Docker support (`Dockerfile`, `Dockerfile.test`, GitHub Actions workflow) indicates the service is intended to run both locally and in containerized environments.
   - `run.sh` is the operational entrypoint for setup, testing, server startup, benchmark runs, and container workflows.

11. **Developer/runtime environment**
   - `.devcontainer` and CI workflows show the repo is designed for reproducible development.
   - The practical runtime stack is Python + FastAPI + Whisper + audio I/O (PyAudio/PortAudio), with the model file bundled in-repo for convenience.

12. **How to draw the architecture**
   - A good diagram should show two ingress paths: **WebSocket/audio stream clients** and **local microphone/demo input**.
   - Both feed into the **stream/session layer**, which calls the **Whisper transcriber** using the bundled model.
   - Results flow back out through the **server callbacks/WebSocket**, while **benchmark/tests** sit alongside the core pipeline as validation and performance verification.

Main entry points: [`whisperflow/streaming.py`](https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/streaming.py) (Streaming), [`whisperflow/fast_server.py`](https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/fast_server.py) (Fast server), [`whisperflow/audio/microphone.py`](https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/audio/microphone.py) (Microphone), [`tests/benchmark/test_benchmark.py`](https://github.com/dimastatz/whisper-flow/blob/main/tests/benchmark/test_benchmark.py) (Benchmarks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core runtime"]
  node_fast_server["Fast server<br/>api server<br/>[fast_server.py]"]
  node_streaming["Streaming<br/>session engine<br/>[streaming.py]"]
  node_transcriber["Transcriber<br/>model layer<br/>[transcriber.py]"]
  node_chat_room["Chat room<br/>session adapter<br/>[chat_room.py]"]
  node_model_file[("Tiny model<br/>bundled model<br/>[tiny.en.pt]")]
  node_pcm_contract["PCM contract<br/>audio format"]
end

subgraph group_ingress["Ingress paths"]
  node_microphone["Microphone<br/>audio capture<br/>[microphone.py]"]
  node_ws_clients(("WebSocket clients<br/>live clients"))
  node_http_clients(("HTTP clients<br/>batch clients"))
  node_mic_users(("Mic users<br/>demo users<br/>[mic_transcribe.py]"))
end

subgraph group_validation["Validation"]
  node_audio_tests["Audio tests<br/>unit tests<br/>[test_audio.py]"]
  node_stream_tests["Stream tests<br/>unit tests<br/>[test_streaming.py]"]
  node_transcriber_tests["Transcriber tests<br/>unit tests"]
  node_chat_tests["Chat tests<br/>unit tests<br/>[test_chat_room.py]"]
  node_benchmarks["Benchmarks<br/>perf suite<br/>[test_benchmark.py]"]
  node_resources["Speech data<br/>benchmark data"]
end

subgraph group_ops["Packaging"]
  node_run_sh["Run script<br/>entrypoint<br/>[run.sh]"]
  node_setup_py["Setup<br/>packaging<br/>[setup.py]"]
  node_requirements["Requirements<br/>deps<br/>[requirements.txt]"]
  node_dockerfile["Dockerfile<br/>container build"]
  node_docker_test["Test image<br/>container build<br/>[Dockerfile.test]"]
  node_ci_workflow["CI workflow<br/>automation<br/>[docker-image.yml]"]
end

node_ws_clients -->|"stream audio"| node_fast_server
node_http_clients -->|"batch PCM"| node_fast_server
node_fast_server -->|"open session"| node_streaming
node_streaming -->|"transcribe"| node_transcriber
node_transcriber -->|"loads model"| node_model_file
node_streaming -->|"emit results"| node_chat_room
node_chat_room -->|"return text"| node_fast_server
node_microphone -->|"capture PCM"| node_pcm_contract
node_mic_users -->|"use demo"| node_microphone
node_microphone -->|"feed chunks"| node_streaming
node_streaming -.->|"expects"| node_pcm_contract
node_audio_tests -.->|"verify"| node_microphone
node_stream_tests -.->|"verify"| node_streaming
node_transcriber_tests -.->|"verify"| node_transcriber
node_chat_tests -.->|"verify"| node_chat_room
node_benchmarks -.->|"measure"| node_streaming
node_benchmarks -.->|"uses data"| node_resources
node_run_sh -->|"start"| node_fast_server
node_run_sh -->|"run perf"| node_benchmarks
node_setup_py -->|"installs deps"| node_requirements
node_dockerfile -->|"package service"| node_fast_server
node_docker_test -->|"test image"| node_benchmarks
node_ci_workflow -->|"builds"| node_dockerfile
node_ci_workflow -->|"builds"| node_docker_test

click node_fast_server "https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/fast_server.py"
click node_streaming "https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/streaming.py"
click node_transcriber "https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/transcriber.py"
click node_chat_room "https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/chat_room.py"
click node_microphone "https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/audio/microphone.py"
click node_model_file "https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/models/tiny.en.pt"
click node_mic_users "https://github.com/dimastatz/whisper-flow/blob/main/tests/examples/mic_transcribe.py"
click node_audio_tests "https://github.com/dimastatz/whisper-flow/blob/main/tests/audio/test_audio.py"
click node_stream_tests "https://github.com/dimastatz/whisper-flow/blob/main/tests/test_streaming.py"
click node_transcriber_tests "https://github.com/dimastatz/whisper-flow/blob/main/tests/test_transcriber.py"
click node_chat_tests "https://github.com/dimastatz/whisper-flow/blob/main/tests/test_chat_room.py"
click node_benchmarks "https://github.com/dimastatz/whisper-flow/blob/main/tests/benchmark/test_benchmark.py"
click node_resources "https://github.com/dimastatz/whisper-flow/blob/main/tests/resources/3081-166546-0000.wav"
click node_run_sh "https://github.com/dimastatz/whisper-flow/blob/main/run.sh"
click node_setup_py "https://github.com/dimastatz/whisper-flow/blob/main/setup.py"
click node_requirements "https://github.com/dimastatz/whisper-flow/blob/main/requirements.txt"
click node_dockerfile "https://github.com/dimastatz/whisper-flow/tree/main/Dockerfile"
click node_docker_test "https://github.com/dimastatz/whisper-flow/blob/main/Dockerfile.test"
click node_ci_workflow "https://github.com/dimastatz/whisper-flow/blob/main/.github/workflows/docker-image.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_fast_server,node_streaming,node_transcriber,node_chat_room,node_model_file,node_pcm_contract toneBlue
class node_microphone,node_ws_clients,node_http_clients,node_mic_users toneAmber
class node_audio_tests,node_stream_tests,node_transcriber_tests,node_chat_tests,node_benchmarks,node_resources toneMint
class node_run_sh,node_setup_py,node_requirements,node_dockerfile,node_docker_test,node_ci_workflow toneRose
```

## Components

### Core runtime

Main transcription service and pipeline

- [Fast server](https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/fast_server.py): api server · FastAPI boundary with WebSocket live transcription, health, and batch HTTP endpoints · `whisperflow/fast_server.py`
- [Streaming](https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/streaming.py): session engine · Orchestrates buffering, incremental updates, and streaming session lifecycle · `whisperflow/streaming.py`
- [Transcriber](https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/transcriber.py): model layer · Loads Whisper and transcribes PCM audio chunks · `whisperflow/transcriber.py`
- [Chat room](https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/chat_room.py): session adapter · Result delivery and session coordination abstraction used by streaming flows · `whisperflow/chat_room.py`
- [Tiny model](https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/models/tiny.en.pt): bundled model · Default lightweight Whisper model shipped with the package · `whisperflow/models/tiny.en.pt`
- **PCM contract**: audio format · 16 kHz mono 16-bit signed PCM input expected by the pipeline

### Ingress paths

External and local audio entry points

- [Microphone](https://github.com/dimastatz/whisper-flow/blob/main/whisperflow/audio/microphone.py): audio capture · Local live audio capture path for demos and interactive use · `whisperflow/audio/microphone.py`
- **WebSocket clients**: live clients · External live transcription clients connected to the server
- **HTTP clients**: batch clients · External batch transcription callers sending PCM chunks
- [Mic users](https://github.com/dimastatz/whisper-flow/blob/main/tests/examples/mic_transcribe.py): demo users · Local users running the microphone example or interactive capture · `tests/examples/mic_transcribe.py`

### Validation

Tests and benchmark coverage

- [Audio tests](https://github.com/dimastatz/whisper-flow/blob/main/tests/audio/test_audio.py): unit tests · Coverage for audio capture and format handling · `tests/audio/test_audio.py`
- [Stream tests](https://github.com/dimastatz/whisper-flow/blob/main/tests/test_streaming.py): unit tests · Coverage for streaming session behavior and result flow · `tests/test_streaming.py`
- [Transcriber tests](https://github.com/dimastatz/whisper-flow/blob/main/tests/test_transcriber.py): unit tests · Coverage for Whisper model loading and transcription behavior · `tests/test_transcriber.py`
- [Chat tests](https://github.com/dimastatz/whisper-flow/blob/main/tests/test_chat_room.py): unit tests · Coverage for chat-room/session coordination · `tests/test_chat_room.py`
- [Benchmarks](https://github.com/dimastatz/whisper-flow/blob/main/tests/benchmark/test_benchmark.py): perf suite · Streaming benchmark checks latency and WER over speech samples · `tests/benchmark/test_benchmark.py`
- [Speech data](https://github.com/dimastatz/whisper-flow/blob/main/tests/resources/3081-166546-0000.wav): benchmark data · Reference audio and transcript sample used by evaluation · `tests/resources/3081-166546-0000.wav`

### Packaging

Build, deploy, and developer tooling

- [Run script](https://github.com/dimastatz/whisper-flow/blob/main/run.sh): entrypoint · Operational script for setup, tests, server startup, and benchmark runs · `run.sh`
- [Setup](https://github.com/dimastatz/whisper-flow/blob/main/setup.py): packaging · Package installation metadata and build configuration · `setup.py`
- [Requirements](https://github.com/dimastatz/whisper-flow/blob/main/requirements.txt): deps · Pinned Python dependencies for runtime and tooling · `requirements.txt`
- [Dockerfile](https://github.com/dimastatz/whisper-flow/tree/main/Dockerfile): container build · Container image for the service runtime · `Dockerfile`
- [Test image](https://github.com/dimastatz/whisper-flow/blob/main/Dockerfile.test): container build · Container image used for test execution · `Dockerfile.test`
- [CI workflow](https://github.com/dimastatz/whisper-flow/blob/main/.github/workflows/docker-image.yml): automation · GitHub Actions workflow for building container images · `.github/workflows/docker-image.yml`

## Connections

- WebSocket clients → Fast server: stream audio · Live clients connect to the WebSocket transcription endpoint
- HTTP clients → Fast server: batch PCM · HTTP callers send chunked PCM for transcription
- Fast server → Streaming: open session · Server delegates live requests into the streaming session engine
- Streaming → Transcriber: transcribe · Session engine invokes the model layer on buffered chunks
- Transcriber → Tiny model: loads model · Whisper transcriber uses the bundled default model when available
- Streaming → Chat room: emit results · Streaming orchestrator sends partial and final results through the session/result adapter
- Chat room → Fast server: return text · Results flow back to the server for client delivery
- Microphone → PCM contract: capture PCM · Local microphone input is normalized to the repo's PCM contract
- Mic users → Microphone: use demo · Example users run the local capture path
- Microphone → Streaming: feed chunks · Captured audio enters the same streaming pipeline as network audio
- Streaming → PCM contract: expects · Streaming logic is built around the fixed PCM format boundary
- Audio tests → Microphone: verify · Tests cover audio input handling
- Stream tests → Streaming: verify · Tests cover session orchestration and incremental result behavior
- Transcriber tests → Transcriber: verify · Tests cover transcription behavior
- Chat tests → Chat room: verify · Tests cover session/result coordination
- Benchmarks → Streaming: measure · Benchmark suite evaluates the streaming segmentation strategy
- Benchmarks → Speech data: uses data · Benchmarking consumes the bundled speech sample and transcript
- Run script → Fast server: start · Operational script launches the service
- Run script → Benchmarks: run perf · Operational script runs benchmark workflows
- Setup → Requirements: installs deps · Packaging pulls in project dependencies
- Dockerfile → Fast server: package service · Container image assembles the runtime around the server
- Test image → Benchmarks: test image · Test container supports validation workflows
- CI workflow → Dockerfile: builds · CI automates container image builds
- CI workflow → Test image: builds · CI also builds the test container image
