# openmoss/moss-tts architecture

> Architecture diagram of the GitHub repository openmoss/moss-tts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openmoss/moss-tts
- Repository: https://github.com/openmoss/moss-tts

## Overview

MOSS-TTS Family is an open-source model suite for speech and sound generation. Its principal workflow is a user submitting text (optionally with reference audio or voice/style instructions) through a model-specific entry point and receiving generated speech or sound; realtime variants stream audio as text arrives. The repository also includes local/llama.cpp inference options and fine-tuning paths. README capabilities guide the overall map; sampled source confirms selected inference and streaming details, so unsampled subsystem wiring is kept conservative.

Main entry points: [`moss_tts_delay/modeling_moss_tts.py`](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/modeling_moss_tts.py) (Delay Model), [`moss_tts_delay/llama_cpp/pipeline.py`](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/llama_cpp/pipeline.py) (llama.cpp Runtime), [`moss_tts_realtime/inferencer.py`](https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/inferencer.py) (Realtime Inference), [`moss_soundeffect_v2/pipeline_moss_soundeffect.py`](https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/pipeline_moss_soundeffect.py) (Sound Pipeline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_speech["Speech Models"]
  node_tts_cli["TTS CLI<br/>[moss_tts_app.py]"]
  node_delay_cli["Delay TTS CLI<br/>[moss_ttsd_app.py]"]
  node_voice_cli["Voice Generator CLI"]
  node_delay_model["Delay Model"]
  node_delay_processor["Delay Processor"]
  node_delay_utils["Delay Inference<br/>[inference_utils.py]"]
  node_local_model["Local Model"]
  node_local15_model["Local v1.5 Model"]
  node_local15_stream["Local Streaming<br/>[streaming.py]"]
end

subgraph group_realtime["Realtime Speech"]
  node_realtime_app["Realtime Demo<br/>[app.py]"]
  node_realtime_api["HTTP Service<br/>[fast_api.py]"]
  node_realtime_infer["Realtime Inference<br/>[inferencer.py]"]
  node_realtime_model["Realtime Model"]
  node_realtime_stream["Stream Processing"]
  node_realtime_client["TTS Session Client<br/>[tts_client.py]"]
end

subgraph group_sound["Sound Generation"]
  node_sound_cli["Sound CLI"]
  node_sound_entry["Sound Inference Entry"]
  node_sound_pipe["Sound Pipeline"]
  node_audio_engine["Audio Diffusion Engine<br/>[wan_audio.py]"]
  node_sound_models["Audio Model Components<br/>[wan_audio_dit.py]"]
end

subgraph group_runtime["Inference Runtime"]
  node_cpp_entry["llama.cpp Runtime<br/>[pipeline.py]"]
  node_cpp_embeddings["Embedding Lookup<br/>[embedding.py]"]
  node_cpp_backbone["llama.cpp Backbone<br/>[backbone.py]"]
end

subgraph group_adapt["Model Adaptation"]
  node_dataset["Fine-tuning Data<br/>[dataset.py]"]
  node_sft["Supervised Fine-tuning<br/>[sft.py]"]
  node_sound_finetune["Sound Fine-tuning<br/>[finetuning.py]"]
end

node_user(("User"))
node_realtime_user(("Streaming Client"))
node_sound_user(("Sound Prompt User"))
node_speech_output["Generated Speech"]
node_audio_output["Generated Sound"]

node_user -->|"submits text"| node_tts_cli
node_tts_cli -->|"requests synthesis"| node_delay_model
node_delay_cli -->|"requests synthesis"| node_delay_model
node_voice_cli -->|"requests synthesis"| node_delay_model
node_delay_processor -->|"prepares inputs"| node_delay_model
node_delay_model -->|"samples tokens"| node_delay_utils
node_delay_model -->|"generates audio"| node_speech_output
node_user -->|"provides text"| node_local_model
node_user -->|"provides text"| node_local15_model
node_local15_model -->|"streams output"| node_local15_stream
node_local_model -->|"generates speech"| node_speech_output
node_local15_stream -->|"emits speech"| node_speech_output
node_realtime_user -->|"submits request"| node_realtime_app
node_realtime_user -->|"starts session"| node_realtime_client
node_realtime_client -->|"sends text chunks"| node_realtime_api
node_realtime_app -->|"requests generation"| node_realtime_infer
node_realtime_api -->|"dispatches inference"| node_realtime_infer
node_realtime_infer -->|"generates tokens"| node_realtime_model
node_realtime_infer -->|"streams audio frames"| node_realtime_stream
node_realtime_stream -->|"returns audio stream"| node_realtime_client
node_sound_user -->|"submits prompt"| node_sound_cli
node_sound_cli -->|"invokes inference"| node_sound_entry
node_sound_user -->|"provides prompt"| node_sound_entry
node_sound_entry -->|"loads and calls"| node_sound_pipe
node_sound_pipe -->|"runs generation"| node_audio_engine
node_audio_engine -->|"uses components"| node_sound_models
node_sound_pipe -->|"returns audio"| node_audio_output
node_user -.->|"submits text"| node_cpp_entry
node_cpp_entry -->|"looks up embeddings"| node_cpp_embeddings
node_cpp_entry -->|"runs backbone"| node_cpp_backbone
node_cpp_entry -->|"decodes speech"| node_speech_output
node_dataset -->|"supplies examples"| node_sft
node_sft -->|"fine-tunes"| node_delay_model
node_dataset -->|"supplies examples"| node_sound_finetune
node_sound_finetune -->|"fine-tunes"| node_sound_models

click node_tts_cli "https://github.com/openmoss/moss-tts/blob/main/clis/moss_tts_app.py"
click node_delay_cli "https://github.com/openmoss/moss-tts/blob/main/clis/moss_ttsd_app.py"
click node_voice_cli "https://github.com/openmoss/moss-tts/blob/main/clis/moss_voice_generator_app.py"
click node_delay_model "https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/modeling_moss_tts.py"
click node_delay_processor "https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/processing_moss_tts.py"
click node_delay_utils "https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/inference_utils.py"
click node_local_model "https://github.com/openmoss/moss-tts/blob/main/moss_tts_local/modeling_moss_tts.py"
click node_local15_model "https://github.com/openmoss/moss-tts/blob/main/moss_tts_local_v1.5/modeling_moss_tts.py"
click node_local15_stream "https://github.com/openmoss/moss-tts/blob/main/moss_tts_local_v1.5/streaming.py"
click node_realtime_app "https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/app.py"
click node_realtime_api "https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/fast_api.py"
click node_realtime_infer "https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/inferencer.py"
click node_realtime_model "https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/mossttsrealtime/modeling_mossttsrealtime.py"
click node_realtime_stream "https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/mossttsrealtime/streaming_mossttsrealtime.py"
click node_sound_cli "https://github.com/openmoss/moss-tts/blob/main/clis/moss_sound_effect_app.py"
click node_sound_entry "https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/infer_from_pipeline.py"
click node_sound_pipe "https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/pipeline_moss_soundeffect.py"
click node_audio_engine "https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/diffsynth/pipelines/wan_audio.py"
click node_sound_models "https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/diffsynth/models/wan_audio_dit.py"
click node_cpp_entry "https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/llama_cpp/pipeline.py"
click node_cpp_embeddings "https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/llama_cpp/embedding.py"
click node_cpp_backbone "https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/llama_cpp/backbone.py"
click node_dataset "https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/finetuning/dataset.py"
click node_sft "https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/finetuning/sft.py"
click node_sound_finetune "https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/finetuning/finetuning.py"
click node_realtime_client "https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/tts_client.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tts_cli,node_delay_cli,node_voice_cli,node_delay_model,node_delay_processor,node_delay_utils,node_local_model,node_local15_model,node_local15_stream,node_user,node_realtime_user,node_sound_user toneBlue
class node_realtime_app,node_realtime_api,node_realtime_infer,node_realtime_model,node_realtime_stream,node_realtime_client toneAmber
class node_sound_cli,node_sound_entry,node_sound_pipe,node_audio_engine,node_sound_models toneMint
class node_cpp_entry,node_cpp_embeddings,node_cpp_backbone toneRose
class node_dataset,node_sft,node_sound_finetune,node_speech_output,node_audio_output toneIndigo
```

## Components

### Speech Models

- [TTS CLI](https://github.com/openmoss/moss-tts/blob/main/clis/moss_tts_app.py): `clis/moss_tts_app.py`
- [Delay TTS CLI](https://github.com/openmoss/moss-tts/blob/main/clis/moss_ttsd_app.py): `clis/moss_ttsd_app.py`
- [Voice Generator CLI](https://github.com/openmoss/moss-tts/blob/main/clis/moss_voice_generator_app.py): `clis/moss_voice_generator_app.py`
- [Delay Model](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/modeling_moss_tts.py): `moss_tts_delay/modeling_moss_tts.py`
- [Delay Processor](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/processing_moss_tts.py): `moss_tts_delay/processing_moss_tts.py`
- [Delay Inference](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/inference_utils.py): `moss_tts_delay/inference_utils.py`
- [Local Model](https://github.com/openmoss/moss-tts/blob/main/moss_tts_local/modeling_moss_tts.py): `moss_tts_local/modeling_moss_tts.py`
- [Local v1.5 Model](https://github.com/openmoss/moss-tts/blob/main/moss_tts_local_v1.5/modeling_moss_tts.py): `moss_tts_local_v1.5/modeling_moss_tts.py`
- [Local Streaming](https://github.com/openmoss/moss-tts/blob/main/moss_tts_local_v1.5/streaming.py): `moss_tts_local_v1.5/streaming.py`

### Realtime Speech

- [Realtime Demo](https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/app.py): `moss_tts_realtime/app.py`
- [HTTP Service](https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/fast_api.py): `moss_tts_realtime/fast_api.py`
- [Realtime Inference](https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/inferencer.py): `moss_tts_realtime/inferencer.py`
- [Realtime Model](https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/mossttsrealtime/modeling_mossttsrealtime.py): `moss_tts_realtime/mossttsrealtime/modeling_mossttsrealtime.py`
- [Stream Processing](https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/mossttsrealtime/streaming_mossttsrealtime.py): `moss_tts_realtime/mossttsrealtime/streaming_mossttsrealtime.py`
- [TTS Session Client](https://github.com/openmoss/moss-tts/blob/main/moss_tts_realtime/tts_client.py): `moss_tts_realtime/tts_client.py`

### Sound Generation

- [Sound CLI](https://github.com/openmoss/moss-tts/blob/main/clis/moss_sound_effect_app.py): `clis/moss_sound_effect_app.py`
- [Sound Inference Entry](https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/infer_from_pipeline.py): `moss_soundeffect_v2/infer_from_pipeline.py`
- [Sound Pipeline](https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/pipeline_moss_soundeffect.py): `moss_soundeffect_v2/pipeline_moss_soundeffect.py`
- [Audio Diffusion Engine](https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/diffsynth/pipelines/wan_audio.py): `moss_soundeffect_v2/diffsynth/pipelines/wan_audio.py`
- [Audio Model Components](https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/diffsynth/models/wan_audio_dit.py): `moss_soundeffect_v2/diffsynth/models/wan_audio_dit.py`

### Inference Runtime

- [llama.cpp Runtime](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/llama_cpp/pipeline.py): `moss_tts_delay/llama_cpp/pipeline.py`
- [Embedding Lookup](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/llama_cpp/embedding.py): `moss_tts_delay/llama_cpp/embedding.py`
- [llama.cpp Backbone](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/llama_cpp/backbone.py): `moss_tts_delay/llama_cpp/backbone.py`

### Model Adaptation

- [Fine-tuning Data](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/finetuning/dataset.py): `moss_tts_delay/finetuning/dataset.py`
- [Supervised Fine-tuning](https://github.com/openmoss/moss-tts/blob/main/moss_tts_delay/finetuning/sft.py): `moss_tts_delay/finetuning/sft.py`
- [Sound Fine-tuning](https://github.com/openmoss/moss-tts/blob/main/moss_soundeffect_v2/finetuning/finetuning.py): `moss_soundeffect_v2/finetuning/finetuning.py`

### Other components

- **User**
- **Streaming Client**
- **Sound Prompt User**
- **Generated Speech**
- **Generated Sound**

## Connections

- User → TTS CLI: submits text
- TTS CLI → Delay Model: requests synthesis
- Delay TTS CLI → Delay Model: requests synthesis
- Voice Generator CLI → Delay Model: requests synthesis
- Delay Processor → Delay Model: prepares inputs
- Delay Model → Delay Inference: samples tokens
- Delay Model → Generated Speech: generates audio
- User → Local Model: provides text
- User → Local v1.5 Model: provides text
- Local v1.5 Model → Local Streaming: streams output
- Local Model → Generated Speech: generates speech
- Local Streaming → Generated Speech: emits speech
- Streaming Client → Realtime Demo: submits request
- Streaming Client → TTS Session Client: starts session
- TTS Session Client → HTTP Service: sends text chunks
- Realtime Demo → Realtime Inference: requests generation
- HTTP Service → Realtime Inference: dispatches inference
- Realtime Inference → Realtime Model: generates tokens
- Realtime Inference → Stream Processing: streams audio frames
- Stream Processing → TTS Session Client: returns audio stream
- Sound Prompt User → Sound CLI: submits prompt
- Sound CLI → Sound Inference Entry: invokes inference
- Sound Prompt User → Sound Inference Entry: provides prompt
- Sound Inference Entry → Sound Pipeline: loads and calls
- Sound Pipeline → Audio Diffusion Engine: runs generation
- Audio Diffusion Engine → Audio Model Components: uses components
- Sound Pipeline → Generated Sound: returns audio
- User → llama.cpp Runtime: submits text
- llama.cpp Runtime → Embedding Lookup: looks up embeddings
- llama.cpp Runtime → llama.cpp Backbone: runs backbone
- llama.cpp Runtime → Generated Speech: decodes speech
- Fine-tuning Data → Supervised Fine-tuning: supplies examples
- Supervised Fine-tuning → Delay Model: fine-tunes
- Fine-tuning Data → Sound Fine-tuning: supplies examples
- Sound Fine-tuning → Audio Model Components: fine-tunes