How openmoss/moss-tts-nano works
1. What this repo is
- This repository packages MOSS-TTS-Nano, a small multilingual text-to-speech system, plus the tooling needed to run it in three ways: direct Python inference, ONNX CPU inference, and a packaged CLI/web demo.
- The codebase is split between runtime/inference, model export, finetuning, and normalization/preprocessing.
2. Core synthesis pipeline
- The central architecture is a two-stage audio-token flow:
- Reference/prompt speech is encoded into discrete audio tokens by the MOSS-Audio-Tokenizer-Nano.
- The TTS model generates new audio-token sequences conditioned on the prompt tokens and input text.
- A decoder converts those tokens back into waveform audio.
- This makes voice cloning the primary usage mode: input is text + prompt audio, output is a generated .wav.
3. Primary inference entrypoints
- infer.py is the direct, non-ONNX local inference path.
- infer_onnx.py is the equivalent path using ONNX Runtime instead of PyTorch.
- Both are designed for one-shot generation and write audio to disk by default.
4. Local demo / service boundary
- app.py provides a local browser demo around the native inference stack.
- app_onnx.py provides the same demo shape but backed by the ONNX runtime.
- The README indicates these run as local HTTP services for interactive testing, so the repo has a clear UI/service boundary above the synthesis engine.
5. Packaged CLI layer
- The moss_tts_nano/ package exposes the installed command-line interface.
- moss_tts_nano/__main__.py and moss_tts_nano/cli.py are the public package boundary.
- The CLI supports at least two top-level modes:
- generate: synthesize a single output file
- serve: launch the local demo/service
- defaults.py centralizes default model and runtime settings.
6. Runtime abstraction
- The repo keeps runtime-specific logic separate from user entrypoints:
- moss_tts_nano_runtime.py for the native/PyTorch-backed path
- onnx_tts_runtime.py for ONNX model execution
- ort_cpu_runtime.py for ONNX Runtime CPU execution details
- This separation is important for the diagram: user-facing commands are thin wrappers over one of two execution backends.
7. ONNX deployment path
- The onnx/ directory contains export scripts for converting Hugging Face-format checkpoints into ONNX assets.
- onnx/export_hf_to_tts_onnx.py produces the TTS-side ONNX files used by infer_onnx.py and app_onnx.py.
- The ONNX path is described as CPU-first but can optionally use CUDA via onnxruntime-gpu.
- This creates a distinct deployment boundary: training/export artifacts on one side, runtime ONNX models on the other.
8. Model acquisition and storage
- The runtime expects model assets either in a local models/ directory or downloaded on first use from Hugging Face / ModelScope.
- The repo ships sample assets in assets/audio, assets/images, and assets/videos, but those are demo inputs/media, not core model state.
- The practical architecture includes a local cache layer for downloaded weights and tokenizer files.
9. Text normalization preprocessing
- text_normalization_pipeline.py and tts_robust_normalizer_single_script.py provide the text normalization layer before synthesis.
- This step is important because the model supports multilingual input and needs stable text formatting for TTS quality.
- In the diagram, normalization sits between raw user text and the model’s text encoder / token generation path.
10. Finetuning subsystem
- finetuning/ is a self-contained training pipeline with its own README, configs, dataset helpers, data preparation, supervised fine-tuning entrypoint, and verification script.
- Key pieces include:
- dataset.py / prepare_data.py for building training data
- sft.py for finetuning
- verify.py for post-training checks
- configs/accelerate_ddp_8gpu.yaml and run_train.sh for distributed training
- This subtree is separate from inference and exists to regenerate or adapt model checkpoints.
11. Data flow for finetuning
- Training data appears to be organized as audio-text pairs in JSONL plus .wav files.
- The flow is: raw dataset audio/text → prepared dataset artifacts → supervised finetuning → verified checkpoint → optional ONNX re-export.
- That means the repo supports a closed loop from training to deployment, but the inference stack does not depend on training code at runtime.
12. Repository-level packaging and execution model
- pyproject.toml and requirements.txt define the installable package and dependencies.
- The repo is meant to be used in a clean Python environment, with editable install enabling the moss-tts-nano command.
- The architecture is intentionally lightweight: local scripts, a package CLI, and optional ONNX runtime rather than a larger service framework.
13. External service boundaries
- External model hosting matters here: the runtime can pull from Hugging Face and ModelScope when local assets are absent.
- For deployment, the README also points to a separate vLLM-Omni serving path for more advanced server features, but that is outside this repository’s core runtime.
14. How to draw the diagram
- Draw two parallel inference paths:
- Native path: user input → normalization → prompt audio encoding → TTS generation → waveform output
- ONNX path: same flow, but through ONNX Runtime and exported model artifacts
- Feed both from the same model/cache layer and the same demo/CLI frontends.
- Put finetuning/export as a separate offline lane that produces the deployable model artifacts consumed by inference.
Main entry points: onnx_tts_runtime.py ONNX Runtime · moss_tts_nano_runtime.py Native Runtime · infer.py Native Infer · sft.py SFT Train
Components 24
Frontends
User-facing scripts and package commands
- CLI · Installed command-line boundary
moss_tts_nano/__main__.py - Commands · Top-level generate and serve commands
moss_tts_nano/cli.py - Defaults · Shared model and runtime defaults
moss_tts_nano/defaults.py - Demo UI · Browser demo on native runtime
app.py - ONNX Demo · Browser demo on ONNX runtime
app_onnx.py
Inference Runtime
Native and ONNX execution paths
- Native Infer · Direct PyTorch inference entrypoint
infer.py - ONNX Infer · ONNX Runtime inference entrypoint
infer_onnx.py - Normalize Text · Text normalization pipeline
text_normalization_pipeline.py - Robust Script · Single-script text normalization
tts_robust_normalizer_single_script.py - Native Runtime · PyTorch-backed synthesis execution
moss_tts_nano_runtime.py - ONNX Runtime · ONNX-backed synthesis execution
onnx_tts_runtime.py - ORT CPU · CPU execution details for ONNX Runtime
ort_cpu_runtime.py
Models & Cache
Local and downloaded assets used by inference
- Audio Tokenizer · Encodes prompt speech into discrete audio tokens
- TTS Model · Generates target audio-token sequences from text and prompt tokens
- Wave Decoder · Converts generated tokens back to waveform
- Model Cache · Local models directory and downloaded weights
- HF / ModelScope · External model hosting used on first use
Finetuning & Export
Training and deployment artifact generation
- ONNX Export · Exports Hugging Face checkpoints to ONNX artifacts
onnx/export_hf_to_tts_onnx.py - Browser Export · Exports ONNX assets for browser/demo deployment
onnx/export_moss_tts_browser_onnx.py - Dataset Prep · Builds training datasets from audio-text pairs
finetuning/dataset.py - Prepare Data · Prepares finetuning artifacts
finetuning/prepare_data.py - SFT Train · Supervised finetuning entrypoint
finetuning/sft.py - Verify · Post-training verification
finetuning/verify.py - Train Script · Distributed training launcher
finetuning/run_train.sh
Connections 29
- CLI → Commands: dispatch · Package entry invokes command router
- Commands → Native Infer: generate · CLI generate mode uses native inference
- Commands → Demo UI: serve · CLI serve mode launches local demo
- Demo UI → Native Runtime: uses · Demo calls native synthesis runtime
- ONNX Demo → ONNX Runtime: uses · Demo calls ONNX synthesis runtime
- Native Infer → Normalize Text: normalize · Normalizes user text before synthesis
- ONNX Infer → Normalize Text: normalize · Normalizes user text before synthesis
- Normalize Text → Robust Script: refines · Shared normalization layer
- Native Infer → Native Runtime: executes · Native entrypoint runs PyTorch-backed synthesis
- ONNX Infer → ONNX Runtime: executes · ONNX entrypoint runs exported model path
- Native Runtime → Audio Tokenizer: encode prompt · Prompt speech becomes audio tokens
- Native Runtime → TTS Model: generate tokens · Conditioned text and prompt tokens drive generation
- Native Runtime → Wave Decoder: decode · Generated tokens are converted to waveform
- ONNX Runtime → Audio Tokenizer: encode prompt · Prompt speech becomes audio tokens
- ONNX Runtime → TTS Model: generate tokens · Conditioned text and prompt tokens drive generation
- ONNX Runtime → Wave Decoder: decode · Generated tokens are converted to waveform
- ONNX Runtime → ORT CPU: backend · ONNX runtime delegates execution details to ORT CPU path
- Model Cache → HF / ModelScope: fetch · Missing local assets can be downloaded from external hosts
- Native Infer → Model Cache: load · Native inference reads local or cached assets
- ONNX Infer → Model Cache: load · ONNX inference reads local or cached assets
- Demo UI → Model Cache: load · Demo loads the same cached assets
- ONNX Demo → Model Cache: load · ONNX demo loads the same cached assets
- Dataset Prep → Prepare Data: prepare · Raw audio-text pairs are turned into training data
- Prepare Data → SFT Train: feed · Prepared artifacts are consumed by finetuning
- Train Script → SFT Train: launch · Shell launcher starts distributed finetuning
- SFT Train → Verify: check · Trained checkpoints are validated
- SFT Train → ONNX Export: export · Finetuned checkpoints can be exported for deployment
- ONNX Export → Browser Export: package · Browser-oriented ONNX assets are derived from exported models
- ONNX Export → ONNX Runtime: deploys to · Exported artifacts are consumed by ONNX inference