# openmoss/moss-tts-nano architecture

> Architecture diagram of the GitHub repository openmoss/moss-tts-nano, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openmoss/moss-tts-nano
- Repository: https://github.com/openmoss/moss-tts-nano

## Overview

1. **What this repo is**
   - This repository packages **MOSS-TTS-Nano**, a small multilingual text-to-speech system, plus the tooling needed to run it in three ways: **direct Python inference**, **ONNX CPU inference**, and a **packaged CLI/web demo**.
   - The codebase is split between **runtime/inference**, **model export**, **finetuning**, and **normalization/preprocessing**.

2. **Core synthesis pipeline**
   - The central architecture is a **two-stage audio-token flow**:
     - **Reference/prompt speech** is encoded into discrete audio tokens by the **MOSS-Audio-Tokenizer-Nano**.
     - The **TTS model** generates new audio-token sequences conditioned on the prompt tokens and input text.
     - A decoder converts those tokens back into waveform audio.
   - This makes voice cloning the primary usage mode: input is **text + prompt audio**, output is a generated `.wav`.

3. **Primary inference entrypoints**
   - `infer.py` is the direct, non-ONNX local inference path.
   - `infer_onnx.py` is the equivalent path using **ONNX Runtime** instead of PyTorch.
   - Both are designed for one-shot generation and write audio to disk by default.

4. **Local demo / service boundary**
   - `app.py` provides a **local browser demo** around the native inference stack.
   - `app_onnx.py` provides the same demo shape but backed by the ONNX runtime.
   - The README indicates these run as local HTTP services for interactive testing, so the repo has a clear **UI/service boundary** above the synthesis engine.

5. **Packaged CLI layer**
   - The `moss_tts_nano/` package exposes the installed command-line interface.
   - `moss_tts_nano/__main__.py` and `moss_tts_nano/cli.py` are the public package boundary.
   - The CLI supports at least two top-level modes:
     - **generate**: synthesize a single output file
     - **serve**: launch the local demo/service
   - `defaults.py` centralizes default model and runtime settings.

6. **Runtime abstraction**
   - The repo keeps runtime-specific logic separate from user entrypoints:
     - `moss_tts_nano_runtime.py` for the native/PyTorch-backed path
     - `onnx_tts_runtime.py` for ONNX model execution
     - `ort_cpu_runtime.py` for ONNX Runtime CPU execution details
   - This separation is important for the diagram: user-facing commands are thin wrappers over one of two execution backends.

7. **ONNX deployment path**
   - The `onnx/` directory contains export scripts for converting Hugging Face-format checkpoints into ONNX assets.
   - `onnx/export_hf_to_tts_onnx.py` produces the TTS-side ONNX files used by `infer_onnx.py` and `app_onnx.py`.
   - The ONNX path is described as **CPU-first** but can optionally use CUDA via `onnxruntime-gpu`.
   - This creates a distinct deployment boundary: **training/export artifacts** on one side, **runtime ONNX models** on the other.

8. **Model acquisition and storage**
   - The runtime expects model assets either in a local `models/` directory or downloaded on first use from **Hugging Face** / **ModelScope**.
   - The repo ships sample assets in `assets/audio`, `assets/images`, and `assets/videos`, but those are demo inputs/media, not core model state.
   - The practical architecture includes a **local cache layer** for downloaded weights and tokenizer files.

9. **Text normalization preprocessing**
   - `text_normalization_pipeline.py` and `tts_robust_normalizer_single_script.py` provide the text normalization layer before synthesis.
   - This step is important because the model supports multilingual input and needs stable text formatting for TTS quality.
   - In the diagram, normalization sits between raw user text and the model’s text encoder / token generation path.

10. **Finetuning subsystem**
   - `finetuning/` is a self-contained training pipeline with its own `README`, configs, dataset helpers, data preparation, supervised fine-tuning entrypoint, and verification script.
   - Key pieces include:
     - `dataset.py` / `prepare_data.py` for building training data
     - `sft.py` for finetuning
     - `verify.py` for post-training checks
     - `configs/accelerate_ddp_8gpu.yaml` and `run_train.sh` for distributed training
   - This subtree is separate from inference and exists to regenerate or adapt model checkpoints.

11. **Data flow for finetuning**
   - Training data appears to be organized as **audio-text pairs** in JSONL plus `.wav` files.
   - The flow is: raw dataset audio/text → prepared dataset artifacts → supervised finetuning → verified checkpoint → optional ONNX re-export.
   - That means the repo supports a closed loop from training to deployment, but the inference stack does not depend on training code at runtime.

12. **Repository-level packaging and execution model**
   - `pyproject.toml` and `requirements.txt` define the installable package and dependencies.
   - The repo is meant to be used in a clean Python environment, with editable install enabling the `moss-tts-nano` command.
   - The architecture is intentionally lightweight: local scripts, a package CLI, and optional ONNX runtime rather than a larger service framework.

13. **External service boundaries**
   - External model hosting matters here: the runtime can pull from **Hugging Face** and **ModelScope** when local assets are absent.
   - For deployment, the README also points to a separate vLLM-Omni serving path for more advanced server features, but that is outside this repository’s core runtime.

14. **How to draw the diagram**
   - Draw two parallel inference paths:
     - **Native path**: user input → normalization → prompt audio encoding → TTS generation → waveform output
     - **ONNX path**: same flow, but through ONNX Runtime and exported model artifacts
   - Feed both from the same **model/cache layer** and the same **demo/CLI frontends**.
   - Put **finetuning/export** as a separate offline lane that produces the deployable model artifacts consumed by inference.

Main entry points: [`onnx_tts_runtime.py`](https://github.com/openmoss/moss-tts-nano/blob/main/onnx_tts_runtime.py) (ONNX Runtime), [`moss_tts_nano_runtime.py`](https://github.com/openmoss/moss-tts-nano/blob/main/moss_tts_nano_runtime.py) (Native Runtime), [`infer.py`](https://github.com/openmoss/moss-tts-nano/blob/main/infer.py) (Native Infer), [`finetuning/sft.py`](https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/sft.py) (SFT Train)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontends["Frontends"]
  node_cli_main["CLI<br/>package entry<br/>[__main__.py]"]
  node_cli["Commands<br/>package cli<br/>[cli.py]"]
  node_defaults["Defaults<br/>runtime config<br/>[defaults.py]"]
  node_app["Demo UI<br/>local app<br/>[app.py]"]
  node_app_onnx["ONNX Demo<br/>local app<br/>[app_onnx.py]"]
end

subgraph group_runtime["Inference Runtime"]
  node_infer["Native Infer<br/>script<br/>[infer.py]"]
  node_infer_onnx["ONNX Infer<br/>script<br/>[infer_onnx.py]"]
  node_text_norm["Normalize Text<br/>preprocess"]
  node_robust_norm["Robust Script<br/>preprocess"]
  node_native_rt["Native Runtime<br/>pytorch runtime"]
  node_onnx_rt["ONNX Runtime"]
  node_ort_cpu["ORT CPU<br/>cpu backend<br/>[ort_cpu_runtime.py]"]
end

subgraph group_models["Models & Cache"]
  node_tokenizer{{"Audio Tokenizer<br/>codec"}}
  node_tts_model{{"TTS Model<br/>generator"}}
  node_decoder{{"Wave Decoder"}}
  node_model_cache[("Model Cache<br/>asset cache")]
  node_hf_ms(("HF / ModelScope<br/>remote source"))
end

subgraph group_offline["Finetuning & Export"]
  node_export_onnx["ONNX Export<br/>export tool"]
  node_export_browser["Browser Export<br/>export tool"]
  node_dataset["Dataset Prep<br/>data prep<br/>[dataset.py]"]
  node_prepare_data["Prepare Data<br/>data prep<br/>[prepare_data.py]"]
  node_sft["SFT Train<br/>training<br/>[sft.py]"]
  node_verify["Verify<br/>check<br/>[verify.py]"]
  node_run_train["Train Script<br/>launcher<br/>[run_train.sh]"]
end

node_cli_main -->|"dispatch"| node_cli
node_cli -->|"generate"| node_infer
node_cli -->|"serve"| node_app
node_app -->|"uses"| node_native_rt
node_app_onnx -->|"uses"| node_onnx_rt
node_infer -->|"normalize"| node_text_norm
node_infer_onnx -->|"normalize"| node_text_norm
node_text_norm -->|"refines"| node_robust_norm
node_infer -->|"executes"| node_native_rt
node_infer_onnx -->|"executes"| node_onnx_rt
node_native_rt -->|"encode prompt"| node_tokenizer
node_native_rt -->|"generate tokens"| node_tts_model
node_native_rt -->|"decode"| node_decoder
node_onnx_rt -->|"encode prompt"| node_tokenizer
node_onnx_rt -->|"generate tokens"| node_tts_model
node_onnx_rt -->|"decode"| node_decoder
node_onnx_rt -->|"backend"| node_ort_cpu
node_model_cache -.->|"fetch"| node_hf_ms
node_infer -->|"load"| node_model_cache
node_infer_onnx -->|"load"| node_model_cache
node_app -->|"load"| node_model_cache
node_app_onnx -->|"load"| node_model_cache
node_dataset -->|"prepare"| node_prepare_data
node_prepare_data -->|"feed"| node_sft
node_run_train -->|"launch"| node_sft
node_sft -->|"check"| node_verify
node_sft -->|"export"| node_export_onnx
node_export_onnx -->|"package"| node_export_browser
node_export_onnx -->|"deploys to"| node_onnx_rt

click node_cli_main "https://github.com/openmoss/moss-tts-nano/blob/main/moss_tts_nano/__main__.py"
click node_cli "https://github.com/openmoss/moss-tts-nano/blob/main/moss_tts_nano/cli.py"
click node_defaults "https://github.com/openmoss/moss-tts-nano/blob/main/moss_tts_nano/defaults.py"
click node_app "https://github.com/openmoss/moss-tts-nano/blob/main/app.py"
click node_app_onnx "https://github.com/openmoss/moss-tts-nano/blob/main/app_onnx.py"
click node_infer "https://github.com/openmoss/moss-tts-nano/blob/main/infer.py"
click node_infer_onnx "https://github.com/openmoss/moss-tts-nano/blob/main/infer_onnx.py"
click node_text_norm "https://github.com/openmoss/moss-tts-nano/blob/main/text_normalization_pipeline.py"
click node_robust_norm "https://github.com/openmoss/moss-tts-nano/blob/main/tts_robust_normalizer_single_script.py"
click node_native_rt "https://github.com/openmoss/moss-tts-nano/blob/main/moss_tts_nano_runtime.py"
click node_onnx_rt "https://github.com/openmoss/moss-tts-nano/blob/main/onnx_tts_runtime.py"
click node_ort_cpu "https://github.com/openmoss/moss-tts-nano/blob/main/ort_cpu_runtime.py"
click node_export_onnx "https://github.com/openmoss/moss-tts-nano/blob/main/onnx/export_hf_to_tts_onnx.py"
click node_export_browser "https://github.com/openmoss/moss-tts-nano/blob/main/onnx/export_moss_tts_browser_onnx.py"
click node_dataset "https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/dataset.py"
click node_prepare_data "https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/prepare_data.py"
click node_sft "https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/sft.py"
click node_verify "https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/verify.py"
click node_run_train "https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/run_train.sh"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli_main,node_cli,node_defaults,node_app,node_app_onnx toneBlue
class node_infer,node_infer_onnx,node_text_norm,node_robust_norm,node_native_rt,node_onnx_rt,node_ort_cpu toneAmber
class node_tokenizer,node_tts_model,node_decoder,node_model_cache,node_hf_ms toneMint
class node_export_onnx,node_export_browser,node_dataset,node_prepare_data,node_sft,node_verify,node_run_train toneRose
```

## Components

### Frontends

User-facing scripts and package commands

- [CLI](https://github.com/openmoss/moss-tts-nano/blob/main/moss_tts_nano/__main__.py): package entry · Installed command-line boundary · `moss_tts_nano/__main__.py`
- [Commands](https://github.com/openmoss/moss-tts-nano/blob/main/moss_tts_nano/cli.py): package cli · Top-level generate and serve commands · `moss_tts_nano/cli.py`
- [Defaults](https://github.com/openmoss/moss-tts-nano/blob/main/moss_tts_nano/defaults.py): runtime config · Shared model and runtime defaults · `moss_tts_nano/defaults.py`
- [Demo UI](https://github.com/openmoss/moss-tts-nano/blob/main/app.py): local app · Browser demo on native runtime · `app.py`
- [ONNX Demo](https://github.com/openmoss/moss-tts-nano/blob/main/app_onnx.py): local app · Browser demo on ONNX runtime · `app_onnx.py`

### Inference Runtime

Native and ONNX execution paths

- [Native Infer](https://github.com/openmoss/moss-tts-nano/blob/main/infer.py): script · Direct PyTorch inference entrypoint · `infer.py`
- [ONNX Infer](https://github.com/openmoss/moss-tts-nano/blob/main/infer_onnx.py): script · ONNX Runtime inference entrypoint · `infer_onnx.py`
- [Normalize Text](https://github.com/openmoss/moss-tts-nano/blob/main/text_normalization_pipeline.py): preprocess · Text normalization pipeline · `text_normalization_pipeline.py`
- [Robust Script](https://github.com/openmoss/moss-tts-nano/blob/main/tts_robust_normalizer_single_script.py): preprocess · Single-script text normalization · `tts_robust_normalizer_single_script.py`
- [Native Runtime](https://github.com/openmoss/moss-tts-nano/blob/main/moss_tts_nano_runtime.py): pytorch runtime · PyTorch-backed synthesis execution · `moss_tts_nano_runtime.py`
- [ONNX Runtime](https://github.com/openmoss/moss-tts-nano/blob/main/onnx_tts_runtime.py): onnx runtime · ONNX-backed synthesis execution · `onnx_tts_runtime.py`
- [ORT CPU](https://github.com/openmoss/moss-tts-nano/blob/main/ort_cpu_runtime.py): cpu backend · CPU execution details for ONNX Runtime · `ort_cpu_runtime.py`

### Models & Cache

Local and downloaded assets used by inference

- **Audio Tokenizer**: codec · Encodes prompt speech into discrete audio tokens
- **TTS Model**: generator · Generates target audio-token sequences from text and prompt tokens
- **Wave Decoder**: decoder · Converts generated tokens back to waveform
- **Model Cache**: asset cache · Local models directory and downloaded weights
- **HF / ModelScope**: remote source · External model hosting used on first use

### Finetuning & Export

Training and deployment artifact generation

- [ONNX Export](https://github.com/openmoss/moss-tts-nano/blob/main/onnx/export_hf_to_tts_onnx.py): export tool · Exports Hugging Face checkpoints to ONNX artifacts · `onnx/export_hf_to_tts_onnx.py`
- [Browser Export](https://github.com/openmoss/moss-tts-nano/blob/main/onnx/export_moss_tts_browser_onnx.py): export tool · Exports ONNX assets for browser/demo deployment · `onnx/export_moss_tts_browser_onnx.py`
- [Dataset Prep](https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/dataset.py): data prep · Builds training datasets from audio-text pairs · `finetuning/dataset.py`
- [Prepare Data](https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/prepare_data.py): data prep · Prepares finetuning artifacts · `finetuning/prepare_data.py`
- [SFT Train](https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/sft.py): training · Supervised finetuning entrypoint · `finetuning/sft.py`
- [Verify](https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/verify.py): check · Post-training verification · `finetuning/verify.py`
- [Train Script](https://github.com/openmoss/moss-tts-nano/blob/main/finetuning/run_train.sh): launcher · Distributed training launcher · `finetuning/run_train.sh`

## Connections

- CLI → Commands: dispatch · Package entry invokes command router
- Commands → Native Infer: generate · CLI generate mode uses native inference
- Commands → Demo UI: serve · CLI serve mode launches local demo
- Demo UI → Native Runtime: uses · Demo calls native synthesis runtime
- ONNX Demo → ONNX Runtime: uses · Demo calls ONNX synthesis runtime
- Native Infer → Normalize Text: normalize · Normalizes user text before synthesis
- ONNX Infer → Normalize Text: normalize · Normalizes user text before synthesis
- Normalize Text → Robust Script: refines · Shared normalization layer
- Native Infer → Native Runtime: executes · Native entrypoint runs PyTorch-backed synthesis
- ONNX Infer → ONNX Runtime: executes · ONNX entrypoint runs exported model path
- Native Runtime → Audio Tokenizer: encode prompt · Prompt speech becomes audio tokens
- Native Runtime → TTS Model: generate tokens · Conditioned text and prompt tokens drive generation
- Native Runtime → Wave Decoder: decode · Generated tokens are converted to waveform
- ONNX Runtime → Audio Tokenizer: encode prompt · Prompt speech becomes audio tokens
- ONNX Runtime → TTS Model: generate tokens · Conditioned text and prompt tokens drive generation
- ONNX Runtime → Wave Decoder: decode · Generated tokens are converted to waveform
- ONNX Runtime → ORT CPU: backend · ONNX runtime delegates execution details to ORT CPU path
- Model Cache → HF / ModelScope: fetch · Missing local assets can be downloaded from external hosts
- Native Infer → Model Cache: load · Native inference reads local or cached assets
- ONNX Infer → Model Cache: load · ONNX inference reads local or cached assets
- Demo UI → Model Cache: load · Demo loads the same cached assets
- ONNX Demo → Model Cache: load · ONNX demo loads the same cached assets
- Dataset Prep → Prepare Data: prepare · Raw audio-text pairs are turned into training data
- Prepare Data → SFT Train: feed · Prepared artifacts are consumed by finetuning
- Train Script → SFT Train: launch · Shell launcher starts distributed finetuning
- SFT Train → Verify: check · Trained checkpoints are validated
- SFT Train → ONNX Export: export · Finetuned checkpoints can be exported for deployment
- ONNX Export → Browser Export: package · Browser-oriented ONNX assets are derived from exported models
- ONNX Export → ONNX Runtime: deploys to · Exported artifacts are consumed by ONNX inference
