How kyutai-labs/delayed-streams-modeling works
Project type and purpose
• This is an open-source toolkit for running Kyutai delayed-streams speech-to-text (STT) and text-to-speech (TTS) models in multiple environments: research (PyTorch), production (Rust server), and on-device (Apple MLX). It bundles client scripts, configuration files, a standalone Rust example, and Jupyter notebooks.
File structure
• Top-level concerns:
• .github / CI & PR workflows
• configs/*.toml → model-server settings for STT/TTS
• scripts/ → Python entry-point scripts for STT & TTS in each environment
• stt-rs/ → standalone Rust example for STT
• stt_pytorch.ipynb, tts_pytorch.ipynb → research notebooks
• audio/ → sample assets
• Key patterns: three implementations per modality (PyTorch, Rust, MLX), each with its own client or server component.
• Configs describe model parameters (batch size, GPU/CUDA flags). Build scripts and Cargo.toml in stt-rs show Rust service packaging.
README for additional insights
• It explains the three implementations, how clients interact with the model (direct PyTorch calls vs. websockets to a moshi-server), and the use cases (real-time streaming, batching, on-device).
• External dependencies: Hugging Face model hub, moshi & moshi-mlx Python packages, moshi-server Rust crate, UV/UVX CLI.