How metavoiceio/metavoice-src works
MetaVoice-1B is a text-to-speech system: users supply text and a reference or preset voice, and receive synthesized speech. Its documented generation stages predict EnCodec tokens with a causal model, predict remaining codebooks with a non-causal model, decode tokens to waveform, and enhance the audio. The repository also provides a web UI, inference server, and first-stage finetuning. Some stage implementations and server wiring were not excerpted, so those connections are included only where the README documents them.
Main entry points: fast_inference.py Fast TTS API · fast_inference_utils.py Generation utilities · training_data.py Training dataset · model.py GPT model
Components 24
User interfaces
- Web UI
app.py - Inference server
serving.py
Speech generation
- Fast TTS API
fam/llm/fast_inference.py - Generation utilities
fam/llm/fast_inference_utils.py - Fast transformer
fam/llm/fast_model.py - Quantization
fam/llm/fast_quantize.py - Inference model
fam/llm/inference.py - GPT model
fam/llm/model.py - Causal generation
fam/llm/mixins/causal.py - Non-causal generation
fam/llm/mixins/non_causal.py - Attention layers
fam/llm/layers/attn.py
Audio and voice
- EnCodec token adapters
fam/llm/adapters/flattened_encodec.py - EnCodec decoder
fam/llm/decoders.py - Audio enhancer
fam/llm/enhancers.py - Speaker encoder
fam/quantiser/audio/speaker_encoder/model.py - Speaker audio features
fam/quantiser/audio/speaker_encoder/audio.py - Text tokenizer
fam/quantiser/text/tokenise.py
Training
- Training preprocessing
fam/llm/preprocessing/audio_token_mode.py - Batch preparation
fam/llm/preprocessing/data_pipeline.py - Training dataset
fam/llm/loaders/training_data.py - First-stage finetuning
fam/llm/finetune.py
Runtime support
- Telemetry client
fam/telemetry/posthog.py
Other components
- User
- PostHog
Connections 27
- User → Web UI: provides text
- User → Inference server: requests synthesis
- Web UI → Fast TTS API: calls synthesis
app.py - Inference server → Fast TTS API: uses inference
serving.py - Fast TTS API → Generation utilities: generates tokens
fam/llm/fast_inference.py - Fast TTS API → Inference model: runs second stage
fam/llm/fast_inference.py - Fast TTS API → EnCodec token adapters: decodes tokens
fam/llm/fast_inference.py - Fast TTS API → Speaker encoder: builds voice model
- Generation utilities → Fast transformer: loads model
fam/llm/fast_inference_utils.py - Generation utilities → Quantization: configures weights
fam/llm/fast_inference_utils.py - Generation utilities → Text tokenizer: loads tokenizer
fam/llm/fast_inference_utils.py - Generation utilities → Speaker encoder: loads embedder
fam/llm/fast_inference_utils.py - Inference model → GPT model: constructs model
fam/llm/inference.py - GPT model → Causal generation: uses generation mixin
fam/llm/model.py - GPT model → Non-causal generation: uses generation mixin
fam/llm/model.py - GPT model → Attention layers: uses attention
fam/llm/model.py - Speaker encoder → Speaker audio features: extracts features
fam/quantiser/audio/speaker_encoder/model.py - Training dataset → Text tokenizer: encodes captions
fam/llm/loaders/training_data.py - Training dataset → Speaker encoder: embeds audio
fam/llm/loaders/training_data.py - Training dataset → Training preprocessing: combines tokens
fam/llm/loaders/training_data.py - Training dataset → Batch preparation: pads tokens
fam/llm/loaders/training_data.py - First-stage finetuning → Training dataset: loads batches
fam/llm/finetune.py - First-stage finetuning → GPT model: trains model
fam/llm/finetune.py - Fast TTS API → EnCodec decoder: decodes waveform
fam/llm/fast_inference.py - EnCodec decoder → Audio enhancer: cleans audio
- Fast TTS API → Telemetry client: records events
fam/llm/fast_inference.py - Telemetry client → PostHog: sends events
fam/telemetry/posthog.py