# fishaudio/fish-speech architecture

> Architecture diagram of the GitHub repository fishaudio/fish-speech, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fishaudio/fish-speech
- Repository: https://github.com/fishaudio/fish-speech

## Overview

Fish Speech is a text-to-speech system for expressive, multilingual speech generation and voice cloning, with S2-Pro and Drama 3 model families. The principal workflow accepts text and optional reference audio through an API, prepares prompts, generates semantic/audio codes, decodes them to audio, and streams the result. The repository also includes a WebUI and model capabilities documented in the README, including multi-speaker and long-form generation; editing and multi-turn revision are documented for Drama 3. Several runtime connections are grounded in imports or excerpts; unsampled UI wiring is not asserted.

Main entry points: [`fish_speech/inference_engine/__init__.py`](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/inference_engine/__init__.py) (Inference Engine), [`tools/server/views.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/views.py) (Request Views), [`fish_speech/models/text2semantic/inference.py`](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/models/text2semantic/inference.py) (Semantic Inference), [`tools/server/api_utils.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/api_utils.py) (API Utilities)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User Interfaces"]
  node_webui["WebUI<br/>[App.tsx]"]
  node_api_client["API Client<br/>[api_client.py]"]
end

subgraph group_serving["API Serving"]
  node_api_server["API Server<br/>[api_server.py]"]
  node_views["Request Views<br/>[views.py]"]
  node_api_utils["API Utilities<br/>[api_utils.py]"]
  node_request_schema["Request Schema<br/>[schema.py]"]
  node_model_manager["Model Manager<br/>[model_manager.py]"]
  node_wrapper["Inference Wrapper<br/>[inference.py]"]
end

subgraph group_generation["Speech Generation"]
  node_engine["Inference Engine<br/>[__init__.py]"]
  node_semantic_inference["Semantic Inference<br/>[inference.py]"]
  node_drama_generation["Drama Generation<br/>[generate.py]"]
  node_audio_tower["Audio Decoder Tower<br/>[tower.py]"]
  node_content_sequence["Content Sequences"]
  node_tokenizer["Fish Tokenizer<br/>[tokenizer.py]"]
end

subgraph group_audio["Audio and Prompts"]
  node_reference_loader["Reference Loader"]
  node_vq_manager["VQ Manager<br/>[vq_manager.py]"]
  node_dac["DAC Codec<br/>[modded_dac.py]"]
  node_audio_files["Audio Files"]
end

node_user(("User"))

node_user -->|"submits text"| node_api_client
node_user -->|"uses"| node_webui
node_api_client -.->|"sends requests"| node_api_server
node_api_server -->|"routes to"| node_views
node_views -->|"calls"| node_api_utils
node_views -->|"uses"| node_model_manager
node_views -->|"calls"| node_wrapper
node_api_utils -->|"uses"| node_engine
node_api_utils -->|"validates"| node_request_schema
node_model_manager -->|"initializes"| node_engine
node_model_manager -->|"loads model"| node_semantic_inference
node_wrapper -->|"consumes results"| node_engine
node_engine -->|"loads prompts"| node_reference_loader
node_engine -->|"decodes codes"| node_vq_manager
node_engine -->|"dispatches generation"| node_semantic_inference
node_semantic_inference -->|"uses"| node_drama_generation
node_semantic_inference -->|"builds sequences"| node_content_sequence
node_content_sequence -->|"encodes text"| node_tokenizer
node_drama_generation -->|"decodes frames"| node_audio_tower
node_reference_loader -->|"reads references"| node_audio_files
node_reference_loader -->|"encodes audio"| node_dac
node_vq_manager -->|"encodes and decodes"| node_dac

click node_webui "https://github.com/fishaudio/fish-speech/blob/main/awesome_webui/src/App.tsx"
click node_api_client "https://github.com/fishaudio/fish-speech/blob/main/tools/api_client.py"
click node_api_server "https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py"
click node_views "https://github.com/fishaudio/fish-speech/blob/main/tools/server/views.py"
click node_api_utils "https://github.com/fishaudio/fish-speech/blob/main/tools/server/api_utils.py"
click node_request_schema "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/utils/schema.py"
click node_model_manager "https://github.com/fishaudio/fish-speech/blob/main/tools/server/model_manager.py"
click node_wrapper "https://github.com/fishaudio/fish-speech/blob/main/tools/server/inference.py"
click node_engine "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/inference_engine/__init__.py"
click node_semantic_inference "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/models/text2semantic/inference.py"
click node_drama_generation "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/models/text2semantic/fish_omni/generate.py"
click node_audio_tower "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/models/text2semantic/fish_omni/tower.py"
click node_content_sequence "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/content_sequence.py"
click node_tokenizer "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/tokenizer.py"
click node_reference_loader "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/inference_engine/reference_loader.py"
click node_vq_manager "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/inference_engine/vq_manager.py"
click node_dac "https://github.com/fishaudio/fish-speech/blob/main/fish_speech/models/dac/modded_dac.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_webui,node_api_client,node_user toneBlue
class node_api_server,node_views,node_api_utils,node_request_schema,node_model_manager,node_wrapper toneAmber
class node_engine,node_semantic_inference,node_drama_generation,node_audio_tower,node_content_sequence,node_tokenizer toneMint
class node_reference_loader,node_vq_manager,node_dac,node_audio_files toneRose
```

## Components

### User Interfaces

- [WebUI](https://github.com/fishaudio/fish-speech/blob/main/awesome_webui/src/App.tsx): `awesome_webui/src/App.tsx`
- [API Client](https://github.com/fishaudio/fish-speech/blob/main/tools/api_client.py): `tools/api_client.py`

### API Serving

- [API Server](https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py): `tools/api_server.py`
- [Request Views](https://github.com/fishaudio/fish-speech/blob/main/tools/server/views.py): `tools/server/views.py`
- [API Utilities](https://github.com/fishaudio/fish-speech/blob/main/tools/server/api_utils.py): `tools/server/api_utils.py`
- [Request Schema](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/utils/schema.py): `fish_speech/utils/schema.py`
- [Model Manager](https://github.com/fishaudio/fish-speech/blob/main/tools/server/model_manager.py): `tools/server/model_manager.py`
- [Inference Wrapper](https://github.com/fishaudio/fish-speech/blob/main/tools/server/inference.py): `tools/server/inference.py`

### Speech Generation

- [Inference Engine](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/inference_engine/__init__.py): `fish_speech/inference_engine/__init__.py`
- [Semantic Inference](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/models/text2semantic/inference.py): `fish_speech/models/text2semantic/inference.py`
- [Drama Generation](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/models/text2semantic/fish_omni/generate.py): `fish_speech/models/text2semantic/fish_omni/generate.py`
- [Audio Decoder Tower](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/models/text2semantic/fish_omni/tower.py): `fish_speech/models/text2semantic/fish_omni/tower.py`
- [Content Sequences](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/content_sequence.py): `fish_speech/content_sequence.py`
- [Fish Tokenizer](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/tokenizer.py): `fish_speech/tokenizer.py`

### Audio and Prompts

- [Reference Loader](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/inference_engine/reference_loader.py): `fish_speech/inference_engine/reference_loader.py`
- [VQ Manager](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/inference_engine/vq_manager.py): `fish_speech/inference_engine/vq_manager.py`
- [DAC Codec](https://github.com/fishaudio/fish-speech/blob/main/fish_speech/models/dac/modded_dac.py): `fish_speech/models/dac/modded_dac.py`
- **Audio Files**

### Other components

- **User**

## Connections

- User → API Client: submits text
- User → WebUI: uses
- API Client → API Server: sends requests
- API Server → Request Views: routes to (evidence: `tools/api_server.py`)
- Request Views → API Utilities: calls (evidence: `tools/server/views.py`)
- Request Views → Model Manager: uses (evidence: `tools/server/views.py`)
- Request Views → Inference Wrapper: calls (evidence: `tools/server/views.py`)
- API Utilities → Inference Engine: uses (evidence: `tools/server/api_utils.py`)
- API Utilities → Request Schema: validates (evidence: `tools/server/api_utils.py`)
- Model Manager → Inference Engine: initializes (evidence: `tools/server/model_manager.py`)
- Model Manager → Semantic Inference: loads model (evidence: `tools/server/model_manager.py`)
- Inference Wrapper → Inference Engine: consumes results (evidence: `tools/server/inference.py`)
- Inference Engine → Reference Loader: loads prompts (evidence: `fish_speech/inference_engine/__init__.py`)
- Inference Engine → VQ Manager: decodes codes (evidence: `fish_speech/inference_engine/__init__.py`)
- Inference Engine → Semantic Inference: dispatches generation (evidence: `fish_speech/inference_engine/__init__.py`)
- Semantic Inference → Drama Generation: uses (evidence: `fish_speech/models/text2semantic/inference.py`)
- Semantic Inference → Content Sequences: builds sequences (evidence: `fish_speech/models/text2semantic/inference.py`)
- Content Sequences → Fish Tokenizer: encodes text (evidence: `fish_speech/content_sequence.py`)
- Drama Generation → Audio Decoder Tower: decodes frames (evidence: `fish_speech/models/text2semantic/fish_omni/generate.py`)
- Reference Loader → Audio Files: reads references (evidence: `fish_speech/inference_engine/reference_loader.py`)
- Reference Loader → DAC Codec: encodes audio (evidence: `fish_speech/inference_engine/reference_loader.py`)
- VQ Manager → DAC Codec: encodes and decodes (evidence: `fish_speech/inference_engine/vq_manager.py`)
