kyutai-labs/pocket-tts

1. Pocket TTS is a single Python package centered on on-device text-to-speech generation. The architecture is built around a small CPU-friendly model, with a clean split between model logic, audio handling, voice conditioning, CLI/server entrypoints, and supporting utilities.

2. The main public entrypoints are:

- pocket_tts/__init__.py and pocket_tts/main.py for the Python API

- pocket_tts/__main__.py plus pocket_tts/main.py for the CLI

- pocket_tts/static/index.html for the local web UI used by the server mode

The same underlying model pipeline is reused by CLI generation, the HTTP server, and library calls.

3. The core orchestration lives in pocket_tts/models/tts_model.py. This is the high-level façade that loads weights, builds the model, prepares voice state from audio or cached state, and exposes audio generation. It is the main boundary between user-facing inputs and the neural inference stack.

4. The model itself is split into two major learned subsystems:

- pocket_tts/models/flow_lm.py: the text-to-audio latent generation / language-model-like component

- pocket_tts/models/mimi.py: the audio codec / representation side used to turn latent predictions into waveform output

The naming and structure suggest a staged pipeline: text conditioning first, then latent/audio generation, then PCM output.

5. Voice cloning is a first-class path. The conditioners/ and data/ layers handle prompt audio and text conditioning:

- conditioners/text.py and conditioners/base.py define conditioning interfaces

- data/audio.py and data/audio_utils.py handle waveform loading, preprocessing, and sample-rate/audio-format concerns

A voice state can come from a raw audio file or from a pre-exported serialized state, which is what enables fast reuse.

6. The repository explicitly supports exporting and reloading voice state for speed. The README’s export_model_state flow maps to the codebase’s stateful inference design and the presence of modules/stateful_module.py. This is important architecturally because long-form or repeated generation avoids recomputing prompt-derived caches.

7. pocket_tts/config/*.yaml holds model variants and language-specific settings. These configs are part of the loading boundary: pretrained language/model variants are selected by config, while custom weights are supported via local YAML paths. The included English, French, German, Italian, Portuguese, and Spanish configs reflect the multi-language deployment surface.

8. pocket_tts/modules/ contains the reusable neural building blocks that make the model portable and CPU-efficient: transformer components, convolution/resampling layers, RoPE, MLPs, layer scaling, SEANet-style audio blocks, and a dummy quantizer. This layer is the implementation substrate for both the flow model and the audio codec.

9. pocket_tts/quantization.py and scripts/evaluate_quantization.py show that quantization is a distinct concern, not an afterthought. The repo treats quantized execution as a performance/size optimization path, which matters for the CPU-first runtime and for maintaining real-time or better-than-real-time generation.

10. pocket_tts/utils/ handles cross-cutting concerns: config parsing, logging, debugging, weight loading, and general helpers. utils/weights_loading.py is especially important because the model is designed to load from packaged weights, local custom configs, and Hugging Face-style voice assets.

11. The project is packaged and deployable in multiple ways:

- Dockerfile, docker-compose.yaml, docker-bake.hcl, and deploy.sh support containerized use

- GitHub Actions workflows automate tests, package publishing, and documentation deployment

- mkdocs.yml and the docs/ tree define the generated documentation site

This makes the repo both a library and an operational application.

12. The test suite covers the main architectural boundaries rather than only unit internals:

- CLI generation behavior

- Python API behavior

- sentence splitting

- quantization

- documentation examples

That tells you the stable contract is the end-to-end TTS pipeline, not just isolated model modules.

13. For an architecture diagram, the most important flow is:

user text + voice prompt → conditioning/audio preprocessing → TTSModel orchestration → stateful latent generation model → audio codec / waveform reconstruction → PCM audio output or WAV file.

The server and CLI are thin wrappers around this same pipeline.

14. The main external dependency boundary is Hugging Face-hosted model/voice assets, plus PyTorch as the runtime. Those assets are not just data files; they are part of the model-loading and voice-cloning workflow, so they should appear as an external model/asset source in any diagram.