How openmoss/moss-tts works
MOSS-TTS Family is an open-source model suite for speech and sound generation. Its principal workflow is a user submitting text (optionally with reference audio or voice/style instructions) through a model-specific entry point and receiving generated speech or sound; realtime variants stream audio as text arrives. The repository also includes local/llama.cpp inference options and fine-tuning paths. README capabilities guide the overall map; sampled source confirms selected inference and streaming details, so unsampled subsystem wiring is kept conservative.
Main entry points: modeling_moss_tts.py Delay Model · pipeline.py llama.cpp Runtime · inferencer.py Realtime Inference · pipeline_moss_soundeffect.py Sound Pipeline
Components 31
Speech Models
- TTS CLI
clis/moss_tts_app.py - Delay TTS CLI
clis/moss_ttsd_app.py - Voice Generator CLI
clis/moss_voice_generator_app.py - Delay Model
moss_tts_delay/modeling_moss_tts.py - Delay Processor
moss_tts_delay/processing_moss_tts.py - Delay Inference
moss_tts_delay/inference_utils.py - Local Model
moss_tts_local/modeling_moss_tts.py - Local v1.5 Model
moss_tts_local_v1.5/modeling_moss_tts.py - Local Streaming
moss_tts_local_v1.5/streaming.py
Realtime Speech
- Realtime Demo
moss_tts_realtime/app.py - HTTP Service
moss_tts_realtime/fast_api.py - Realtime Inference
moss_tts_realtime/inferencer.py - Realtime Model
moss_tts_realtime/mossttsrealtime/modeling_mossttsrealtime.py - Stream Processing
moss_tts_realtime/mossttsrealtime/streaming_mossttsrealtime.py - TTS Session Client
moss_tts_realtime/tts_client.py
Sound Generation
- Sound CLI
clis/moss_sound_effect_app.py - Sound Inference Entry
moss_soundeffect_v2/infer_from_pipeline.py - Sound Pipeline
moss_soundeffect_v2/pipeline_moss_soundeffect.py - Audio Diffusion Engine
moss_soundeffect_v2/diffsynth/pipelines/wan_audio.py - Audio Model Components
moss_soundeffect_v2/diffsynth/models/wan_audio_dit.py
Inference Runtime
- llama.cpp Runtime
moss_tts_delay/llama_cpp/pipeline.py - Embedding Lookup
moss_tts_delay/llama_cpp/embedding.py - llama.cpp Backbone
moss_tts_delay/llama_cpp/backbone.py
Model Adaptation
- Fine-tuning Data
moss_tts_delay/finetuning/dataset.py - Supervised Fine-tuning
moss_tts_delay/finetuning/sft.py - Sound Fine-tuning
moss_soundeffect_v2/finetuning/finetuning.py
Other components
- User
- Streaming Client
- Sound Prompt User
- Generated Speech
- Generated Sound
Connections 35
- User → TTS CLI: submits text
- TTS CLI → Delay Model: requests synthesis
- Delay TTS CLI → Delay Model: requests synthesis
- Voice Generator CLI → Delay Model: requests synthesis
- Delay Processor → Delay Model: prepares inputs
- Delay Model → Delay Inference: samples tokens
- Delay Model → Generated Speech: generates audio
- User → Local Model: provides text
- User → Local v1.5 Model: provides text
- Local v1.5 Model → Local Streaming: streams output
- Local Model → Generated Speech: generates speech
- Local Streaming → Generated Speech: emits speech
- Streaming Client → Realtime Demo: submits request
- Streaming Client → TTS Session Client: starts session
- TTS Session Client → HTTP Service: sends text chunks
- Realtime Demo → Realtime Inference: requests generation
- HTTP Service → Realtime Inference: dispatches inference
- Realtime Inference → Realtime Model: generates tokens
- Realtime Inference → Stream Processing: streams audio frames
- Stream Processing → TTS Session Client: returns audio stream
- Sound Prompt User → Sound CLI: submits prompt
- Sound CLI → Sound Inference Entry: invokes inference
- Sound Prompt User → Sound Inference Entry: provides prompt
- Sound Inference Entry → Sound Pipeline: loads and calls
- Sound Pipeline → Audio Diffusion Engine: runs generation
- Audio Diffusion Engine → Audio Model Components: uses components
- Sound Pipeline → Generated Sound: returns audio
- User → llama.cpp Runtime: submits text
- llama.cpp Runtime → Embedding Lookup: looks up embeddings
- llama.cpp Runtime → llama.cpp Backbone: runs backbone
- llama.cpp Runtime → Generated Speech: decodes speech
- Fine-tuning Data → Supervised Fine-tuning: supplies examples
- Supervised Fine-tuning → Delay Model: fine-tunes
- Fine-tuning Data → Sound Fine-tuning: supplies examples
- Sound Fine-tuning → Audio Model Components: fine-tunes