How lucasjinreal/kokoros works
Project type and purpose
• This is a Rust-based TTS (Text-to-Speech) engine packaged as:
a) A core library crate (kokoros) implementing phonemization, tokenization, ONNX‐based inference, audio I/O, etc.
b) A CLI binary crate (koko) that wraps the core TTS engine for local use (text/file/stream) and also exposes an OpenAI‐compatible HTTP server via a separate crate (kokoros-openai).
c) Shell/Python scripts under scripts/ to download models, voices, and run OpenAI‐style client demos.
d) A Dockerfile for containerized usage.
File structure
• Top‐level crates:
• koko/ (CLI binary)
• kokoros/ (core engine library)
• kokoros-openai/ (server adapter)
• Core engine modules under kokoros/src/:
• onn/ – ONNX runtime backends for different model flavors (koko, yolo, base)
• tts/ – text‐to‐speech pipeline (tokenizer, phonemizer, style mixing, normalization, vocabulary)
• utils/ – file I/O, WAV/MP3 encoding
• Data directories:
• checkpoints/ and data/ (model weights, voices.json)
• scripts/ for fetching these assets
• Build & deploy:
• Cargo.toml, Dockerfile, download_all.sh, install.sh
README for additional insights
• Describes CLI commands: koko text, koko file, koko stream, koko openai
• Mentions streaming mode, OpenAI-compatible server, Docker usage
• Emphasizes extremely fast inference via ONNX
1) User Interfaces
• CLI (koko binary)
• HTTP API (OpenAI-compatible server, via kokoros-openai)
• Streaming interface (stdin→stdout)
2) Core TTS Engine (kokoros library)
• Tokenizer
• Phonemizer
• Normalizer
• Style mixer
3) ONNX Runtime Backends (kokoros/src/onn)
• ORT Koko model
• ORT Yolo model
• ORT base model
4) Utilities (utils)
• File I/O
• WAV/MP3 encoding
5) Data Storage
• Model checkpoints (checkpoints/, data/)
• Voice metadata (voices.json)
6) External Dependencies
• HuggingFace (model fetching)
• Python scripts (scripts/)
• Docker
• CLI/HTTP API → parses user request → calls Core TTS Engine
• Core TTS Engine → invokes Tokenizer → Phonemizer → Normalizer → passes token IDs & style params to ONNX Runtime
• ONNX Runtime → loads model weights from checkpoints/ → produces raw waveform data
• Raw waveform → Utilities module → encodes to WAV/MP3 → writes to disk or streams to stdout/HTTP response
• While fetching models: Python scripts → download from HuggingFace → populate data/ & checkpoints/
c. Architectural patterns & design principles
• Modular Rust crates for separation of concerns (CLI, core engine, server adapter)
• Plug-in backends for multiple ONNX model formats
• Clear layering: UI → engine → runtime → I/O
• Data‐driven design: external checkpoints & metadata
d. Key technologies and frameworks
• Rust (cargo workspace)
• ONNX Runtime for inference
• Rust audio libraries (WAV/MP3)
• HTTP framework (e.g., Actix or Rocket via kokoros-openai)
• Docker for containerization
• Python for preprocessing and model downloading