# kawshikbuet17/orpheus-tts-bengali-finetuning architecture

> Architecture diagram of the GitHub repository kawshikbuet17/orpheus-tts-bengali-finetuning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kawshikbuet17/orpheus-tts-bengali-finetuning
- Repository: https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning

## Overview

1. **Repo purpose and top-level split**
   - This repository is organized around one core product: a text-to-speech model family called Orpheus.
   - The codebase is split into three main concerns: **model runtime/inference**, **training (pretrain + finetune)**, and **deployment/examples**.

2. **Core inference package boundary**
   - The packaged runtime lives in `orpheus_tts_pypi/orpheus_tts`.
   - `engine_class.py` and `decoder.py` appear to define the public model interface and the generation/decoding path.
   - This is the boundary most users hit when they import `OrpheusModel` and call speech generation.

3. **Published package vs local repository code**
   - The repo includes a PyPI-packaged distribution (`orpheus_tts_pypi`) with packaging metadata in `pyproject.toml` and `setup.py`.
   - The README explicitly notes that some users may need to import from the local package copy instead of the published `orpheus-speech` package when fixes have not yet been released.
   - For architecture diagrams, treat this as a **distribution boundary**: source repo code vs installed package.

4. **Inference data flow**
   - The primary inference flow is: **text prompt + voice selection + generation parameters → streamed audio chunks → WAV/file/client playback**.
   - The README’s example writes mono 24 kHz PCM audio incrementally as chunks arrive, which indicates the model generates **streaming audio tokens/frames**, not a single blocking output.
   - This streaming behavior is central to the low-latency claim and is likely the main runtime path in the package.

5. **Model hosting and external inference options**
   - The repo includes an official Baseten example under `additional_inference_options/baseten_inference_example`, indicating a production deployment path outside the local Python package.
   - That path is a separate inference boundary: **client code → remote hosted Orpheus endpoint → streaming audio response**.
   - The README treats Baseten as the preferred production inference partner.

6. **No-GPU inference alternative**
   - `additional_inference_options/no_gpu` provides a CPU/LLM.cpp-style fallback path for running without CUDA/GPU.
   - Architecturally, this is an alternative runtime backend rather than a separate product: same TTS task, different execution engine and hardware assumptions.

7. **Audio post-processing / watermarking**
   - `additional_inference_options/watermark_audio` adds an optional audio post-processing stage called Silent Cipher watermarking.
   - This is not part of the core model; it is a downstream audio transform applied after generation, so it belongs as an optional **post-inference processing branch**.

8. **Realtime streaming demo**
   - `realtime_streaming_example` contains a Python server (`main.py`) and browser client (`client.html`).
   - This forms a small end-to-end streaming system: **browser UI → local streaming server → Orpheus inference → live audio playback in the browser**.
   - It demonstrates the model’s low-latency behavior and the integration point between model output and client consumption.

9. **Training subsystem: pretraining**
   - `pretrain/` contains a standalone training pipeline for the base model, with its own `config.yaml`, `readme.md`, and `train.py`.
   - This path is for large-scale model training from broader text/speech corpora and is conceptually separate from speaker-specific finetuning.
   - The README describes it as analogous to LLM training with long sequences and chained inputs.

10. **Training subsystem: finetuning**
   - `finetune/` contains the finetuning pipeline, including `train.py`, `lora.py`, and multiple configs for standard and Bengali variants.
   - The directory also includes scripts for full training and QLoRA-based training, showing that finetuning supports multiple adaptation strategies.
   - The likely flow is: **prepared HF dataset → config-driven training script → LoRA/QLoRA or full finetune → checkpoint**.

11. **Data preparation / preprocessing**
   - `preprocess_bengali_telecom_orpheus_local.py` is a data preparation script for a specific Bengali telecom dataset flow.
   - This indicates a preprocessing boundary between raw audio/text sources and the training dataset format consumed by the finetuning pipeline.
   - The README also references Hugging Face datasets as the canonical training input shape.

12. **Evaluation and testing**
   - `test_orpheus_repo.py` and `test_orpheus_concurrent.py` provide validation around repository behavior and concurrency, which is especially relevant for streaming generation.
   - `orpheus_benchmark_ui.py` suggests an internal benchmarking interface for comparing latency/throughput or generation quality across configurations.
   - These are supporting systems around the inference path rather than part of the core model.

13. **Configuration and environment**
   - Root-level `requirements.txt`, `requirements_finetune.txt`, `.python-version`, and multiple YAML configs show that the repo is managed as a Python-first project with separate runtime and training dependency sets.
   - The README references `transformers`, `datasets`, `wandb`, `trl`, `flash_attn`, `torch`, `accelerate`, and `vllm`, which materially shape the architecture: training is Hugging Face/Accelerate-based, and fast inference is vLLM-backed.

14. **What to show in an architecture diagram**
   - A clear diagram should show four layers: **clients/examples**, **model runtime package**, **training pipelines**, and **optional deployment/post-processing integrations**.
   - The most important arrows are: **prompt/dataset input → model generation/training → streamed audio/checkpoint output**, with separate branches for **Baseten hosting**, **CPU fallback**, and **watermarking**.

Main entry points: [`orpheus_tts_pypi/orpheus_tts/engine_class.py`](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_tts_pypi/orpheus_tts/engine_class.py) (Engine), [`finetune/train.py`](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/finetune/train.py) (Finetune), [`orpheus_tts_pypi/pyproject.toml`](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_tts_pypi/pyproject.toml) (PyPI package), [`orpheus_tts_pypi/orpheus_tts/decoder.py`](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_tts_pypi/orpheus_tts/decoder.py) (Decoder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_distribution["Distribution boundary"]
  node_n_repo["Repo source<br/>monorepo"]
  node_n_pkg_boundary["PyPI package<br/>distribution<br/>[pyproject.toml]"]
end

subgraph group_g_runtime["Inference runtime"]
  node_n_engine["Engine<br/>model API<br/>[engine_class.py]"]
  node_n_decoder["Decoder<br/>streaming decode<br/>[decoder.py]"]
  node_n_stream_api["Stream output<br/>audio stream"]
end

subgraph group_g_training["Training pipelines"]
  node_n_pretrain["Pretrain<br/>base training<br/>[train.py]"]
  node_n_finetune["Finetune<br/>adaptation training<br/>[train.py]"]
  node_n_lora["LoRA utils<br/>adaptation helper<br/>[lora.py]"]
  node_n_data_prep["Data prep<br/>preprocess"]
  node_n_checkpoint[("Checkpoint<br/>trained weights")]
end

subgraph group_g_examples["Deployment and examples"]
  node_n_realtime_server["Realtime server<br/>demo server<br/>[main.py]"]
  node_n_browser_client["Browser client<br/>web demo<br/>[client.html]"]
  node_n_baseten["Baseten demo<br/>remote inference<br/>[call_orpheus.py]"]
  node_n_cpu_fallback["CPU fallback<br/>alt runtime<br/>[README.md]"]
  node_n_watermark["Watermarking<br/>audio postprocess<br/>[watermark.py]"]
end

subgraph group_g_validation["Validation"]
  node_n_benchmark["Benchmark UI<br/>perf tool"]
  node_n_test_repo["Repo test<br/>validation test"]
  node_n_test_conc["Concurrency test<br/>validation test"]
end

node_n_repo -->|"packages"| node_n_pkg_boundary
node_n_pkg_boundary -->|"exposes"| node_n_engine
node_n_engine -->|"uses"| node_n_decoder
node_n_decoder -->|"emits"| node_n_stream_api
node_n_realtime_server -->|"calls"| node_n_engine
node_n_browser_client -->|"connects"| node_n_realtime_server
node_n_baseten -->|"receives"| node_n_stream_api
node_n_cpu_fallback -.->|"alternative backend"| node_n_engine
node_n_watermark -.->|"post-processes"| node_n_stream_api
node_n_data_prep -->|"feeds"| node_n_finetune
node_n_pretrain -->|"produces"| node_n_checkpoint
node_n_finetune -->|"uses"| node_n_lora
node_n_lora -->|"produces"| node_n_checkpoint
node_n_finetune -->|"produces"| node_n_checkpoint
node_n_test_repo -.->|"validates"| node_n_engine
node_n_test_conc -.->|"validates"| node_n_stream_api
node_n_benchmark -.->|"measures"| node_n_engine
node_n_checkpoint -->|"loads into"| node_n_engine

click node_n_pkg_boundary "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_tts_pypi/pyproject.toml"
click node_n_engine "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_tts_pypi/orpheus_tts/engine_class.py"
click node_n_decoder "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_tts_pypi/orpheus_tts/decoder.py"
click node_n_realtime_server "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/realtime_streaming_example/main.py"
click node_n_browser_client "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/realtime_streaming_example/client.html"
click node_n_baseten "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/additional_inference_options/baseten_inference_example/call_orpheus.py"
click node_n_cpu_fallback "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/additional_inference_options/no_gpu/README.md"
click node_n_watermark "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/additional_inference_options/watermark_audio/watermark.py"
click node_n_pretrain "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/pretrain/train.py"
click node_n_finetune "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/finetune/train.py"
click node_n_lora "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/finetune/lora.py"
click node_n_data_prep "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/preprocess_bengali_telecom_orpheus_local.py"
click node_n_benchmark "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_benchmark_ui.py"
click node_n_test_repo "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/test_orpheus_repo.py"
click node_n_test_conc "https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/test_orpheus_concurrent.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_repo,node_n_pkg_boundary toneBlue
class node_n_engine,node_n_decoder,node_n_stream_api toneAmber
class node_n_pretrain,node_n_finetune,node_n_lora,node_n_data_prep,node_n_checkpoint toneMint
class node_n_realtime_server,node_n_browser_client,node_n_baseten,node_n_cpu_fallback,node_n_watermark toneRose
class node_n_benchmark,node_n_test_repo,node_n_test_conc toneIndigo
```

## Components

### Distribution boundary

Source repository vs packaged TTS runtime

- **Repo source**: monorepo · Top-level repository coordinating runtime, training, and examples
- [PyPI package](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_tts_pypi/pyproject.toml): distribution · Packaged Orpheus TTS runtime published via setup/pyproject · `orpheus_tts_pypi/pyproject.toml`

### Inference runtime

Core streaming generation path

- [Engine](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_tts_pypi/orpheus_tts/engine_class.py): model API · Public model interface for inference calls · `orpheus_tts_pypi/orpheus_tts/engine_class.py`
- [Decoder](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_tts_pypi/orpheus_tts/decoder.py): streaming decode · Generation and audio token decoding path · `orpheus_tts_pypi/orpheus_tts/decoder.py`
- **Stream output**: audio stream · Incremental audio chunks consumed by clients

### Training pipelines

Pretraining, finetuning, and data preparation

- [Pretrain](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/pretrain/train.py): base training · Standalone large-scale base model training pipeline · `pretrain/train.py`
- [Finetune](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/finetune/train.py): adaptation training · LoRA, QLoRA, and full finetuning pipelines · `finetune/train.py`
- [LoRA utils](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/finetune/lora.py): adaptation helper · Parameter-efficient finetuning support · `finetune/lora.py`
- [Data prep](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/preprocess_bengali_telecom_orpheus_local.py): preprocess · Dataset preparation for Bengali telecom training flow · `preprocess_bengali_telecom_orpheus_local.py`
- **Checkpoint**: trained weights · Saved finetuned or pretrained model artifact

### Deployment and examples

Local demos and external inference options

- [Realtime server](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/realtime_streaming_example/main.py): demo server · Local streaming server for browser playback · `realtime_streaming_example/main.py`
- [Browser client](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/realtime_streaming_example/client.html): web demo · Client UI that receives and plays live audio · `realtime_streaming_example/client.html`
- [Baseten demo](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/additional_inference_options/baseten_inference_example/call_orpheus.py): remote inference · Hosted production inference example and client wrapper · `additional_inference_options/baseten_inference_example/call_orpheus.py`
- [CPU fallback](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/additional_inference_options/no_gpu/README.md): alt runtime · No-GPU inference path using a CPU-oriented backend · `additional_inference_options/no_gpu/README.md`
- [Watermarking](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/additional_inference_options/watermark_audio/watermark.py): audio postprocess · Optional Silent Cipher watermark transform after generation · `additional_inference_options/watermark_audio/watermark.py`

### Validation

Tests and benchmarking

- [Benchmark UI](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/orpheus_benchmark_ui.py): perf tool · Interface for comparing latency, throughput, and output behavior · `orpheus_benchmark_ui.py`
- [Repo test](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/test_orpheus_repo.py): validation test · Behavior checks for repository-level inference flows · `test_orpheus_repo.py`
- [Concurrency test](https://github.com/kawshikbuet17/orpheus-tts-bengali-finetuning/blob/dev/test_orpheus_concurrent.py): validation test · Checks streaming and concurrent generation behavior · `test_orpheus_concurrent.py`

## Connections

- Repo source → PyPI package: packages · Source repository ships the runtime distribution
- PyPI package → Engine: exposes · Published package exposes the model API
- Engine → Decoder: uses · Inference calls flow into decoding and generation
- Decoder → Stream output: emits · Decoder produces streamed audio chunks
- Realtime server → Engine: calls · Local demo invokes the packaged model runtime
- Browser client → Realtime server: connects · Browser client consumes streamed audio from the server
- Baseten demo → Stream output: receives · Remote hosted inference returns streamed audio responses
- CPU fallback → Engine: alternative backend · CPU path swaps the execution backend for no-GPU operation
- Watermarking → Stream output: post-processes · Optional transform runs after audio generation
- Data prep → Finetune: feeds · Prepared datasets are consumed by finetuning
- Pretrain → Checkpoint: produces · Base training outputs model checkpoints
- Finetune → LoRA utils: uses · Finetuning can apply LoRA/QLoRA adaptation
- LoRA utils → Checkpoint: produces · Adaptation pipeline writes finetuned checkpoints
- Finetune → Checkpoint: produces · Full finetune also emits checkpoints
- Repo test → Engine: validates · Repository tests exercise the main inference path
- Concurrency test → Stream output: validates · Concurrency tests focus on streaming behavior
- Benchmark UI → Engine: measures · Benchmark UI compares runtime behavior and latency
- Checkpoint → Engine: loads into · Trained weights are loaded by the inference engine