# lucasjinreal/kokoros architecture

> Architecture diagram of the GitHub repository lucasjinreal/kokoros, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lucasjinreal/kokoros
- Repository: https://github.com/lucasjinreal/kokoros

## Overview

##### Project type and purpose
- This is a Rust-based TTS (Text-to-Speech) engine packaged as:
a) A core library crate (`kokoros`) implementing phonemization, tokenization, ONNX‐based inference, audio I/O, etc.
b) A CLI binary crate (`koko`) that wraps the core TTS engine for local use (text/file/stream) and also exposes an OpenAI‐compatible HTTP server via a separate crate (`kokoros-openai`).
c) Shell/Python scripts under `scripts/` to download models, voices, and run OpenAI‐style client demos.
d) A Dockerfile for containerized usage.

##### File structure
- Top‐level crates:
- `koko/` (CLI binary)
- `kokoros/` (core engine library)
- `kokoros-openai/` (server adapter)
- Core engine modules under `kokoros/src/`:
- `onn/` – ONNX runtime backends for different model flavors (koko, yolo, base)
- `tts/` – text‐to‐speech pipeline (tokenizer, phonemizer, style mixing, normalization, vocabulary)
- `utils/` – file I/O, WAV/MP3 encoding
- Data directories:
- `checkpoints/` and `data/` (model weights, voices.json)
- `scripts/` for fetching these assets
- Build & deploy:
- `Cargo.toml`, `Dockerfile`, `download_all.sh`, `install.sh`

##### README for additional insights
- Describes CLI commands: `koko text`, `koko file`, `koko stream`, `koko openai`
- Mentions streaming mode, OpenAI-compatible server, Docker usage
- Emphasizes extremely fast inference via ONNX

1) User Interfaces
- CLI (`koko` binary)
- HTTP API (OpenAI-compatible server, via `kokoros-openai`)
- Streaming interface (stdin→stdout)
2) Core TTS Engine (`kokoros` library)
- Tokenizer
- Phonemizer
- Normalizer
- Style mixer
3) ONNX Runtime Backends (`kokoros/src/onn`)
- ORT Koko model
- ORT Yolo model
- ORT base model
4) Utilities (`utils`)
- File I/O
- WAV/MP3 encoding
5) Data Storage
- Model checkpoints (`checkpoints/`, `data/`)
- Voice metadata (`voices.json`)
6) External Dependencies
- HuggingFace (model fetching)
- Python scripts (`scripts/`)
- Docker

- CLI/HTTP API → parses user request → calls Core TTS Engine
- Core TTS Engine → invokes Tokenizer → Phonemizer → Normalizer → passes token IDs & style params to ONNX Runtime
- ONNX Runtime → loads model weights from `checkpoints/` → produces raw waveform data
- Raw waveform → Utilities module → encodes to WAV/MP3 → writes to disk or streams to stdout/HTTP response
- While fetching models: Python scripts → download from HuggingFace → populate `data/` & `checkpoints/`

c. Architectural patterns & design principles
- Modular Rust crates for separation of concerns (CLI, core engine, server adapter)
- Plug-in backends for multiple ONNX model formats
- Clear layering: UI → engine → runtime → I/O
- Data‐driven design: external checkpoints & metadata

d. Key technologies and frameworks
- Rust (cargo workspace)
- ONNX Runtime for inference
- Rust audio libraries (WAV/MP3)
- HTTP framework (e.g., Actix or Rocket via `kokoros-openai`)
- Docker for containerization
- Python for preprocessing and model downloading

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interfaces
    subgraph "User Interfaces"
        direction TB
        CLI["CLI (koko)"]:::ui
        HTTPAPI["HTTP API Server"]:::ui
        CLI_STREAM["Streaming Interface"]:::ui
    end

    %% Core TTS Pipeline
    subgraph "Core TTS Engine Library" 
        direction TB
        CoreEngine["Core Library (kokoros)"]:::core
        Tokenize["Tokenizer"]:::core
        Phonemizer["Phonemizer"]:::core
        Normalize["Normalizer"]:::core
        StyleMixer["Style Mixer"]:::core
    end

    %% ONNX Runtime Backends
    subgraph "ONNX Runtime Backends"
        direction TB
        ORT_Base["ORT Base Model"]:::backend
        ORT_Koko["ORT Koko Model"]:::backend
        ORT_Yolo["ORT Yolo Model"]:::backend
    end

    %% Utilities
    subgraph "Utilities"
        direction TB
        FileIO["File I/O"]:::util
        WAV["WAV Encoding"]:::util
        MP3["MP3 Encoding"]:::util
    end

    %% Data Storage
    subgraph "Data Storage"
        direction TB
        Checkpoints["Model Checkpoints"]:::storage
        DataMeta["Voice Metadata"]:::storage
    end

    %% External Dependencies
    subgraph "External Services & Scripts"
        direction TB
        HF["Hugging Face"]:::external
        ScriptsDL["Download Scripts"]:::external
        ScriptDemo["OpenAI Client Demo"]:::external
    end

    %% Containerization
    subgraph "Containerization & Setup"
        direction TB
        Docker["Dockerfile"]:::external
        DownloadAll["download_all.sh"]:::external
        Install["install.sh"]:::external
    end

    %% Connections
    CLI -->|"text/file/stream request"| CoreEngine
    HTTPAPI -->|"HTTP request (port 3000)"| CoreEngine
    CLI_STREAM -->|"stdin→stdout"| CoreEngine

    CoreEngine -->|"text → tokens"| Tokenize
    Tokenize -->|"tokens → phonemes"| Phonemizer
    Phonemizer -->|"phonemes → normalized text"| Normalize
    Normalize -->|"normalized tokens + style"| StyleMixer
    StyleMixer -->|"token IDs + style params"| ORT_Base
    StyleMixer -->|"token IDs + style params"| ORT_Koko
    StyleMixer -->|"token IDs + style params"| ORT_Yolo

    ORT_Base -->|"loads weights"| Checkpoints
    ORT_Koko -->|"loads weights"| Checkpoints
    ORT_Yolo -->|"loads weights"| Checkpoints

    ORT_Base -->|"tensor → waveform"| FileIO
    ORT_Koko -->|"tensor → waveform"| FileIO
    ORT_Yolo -->|"tensor → waveform"| FileIO

    FileIO --> WAV
    FileIO --> MP3

    WAV -->|"audio bytes"| CLI
    WAV -->|"audio bytes"| HTTPAPI
    MP3 -->|"audio bytes"| CLI
    MP3 -->|"audio bytes"| HTTPAPI

    ScriptsDL -.->|"downloads models/voices"| Checkpoints
    ScriptsDL -.-> DataMeta
    HF -.->|"model assets"| ScriptsDL

    Docker --> CLI
    Docker --> HTTPAPI
    DownloadAll --> ScriptsDL
    Install --> CLI

    ScriptDemo --> HTTPAPI

    %% Click Events
    click CLI "https://github.com/lucasjinreal/kokoros/blob/main/koko/Cargo.toml"
    click CLI "https://github.com/lucasjinreal/kokoros/blob/main/koko/src/main.rs"
    click HTTPAPI "https://github.com/lucasjinreal/kokoros/blob/main/kokoros-openai/Cargo.toml"
    click HTTPAPI "https://github.com/lucasjinreal/kokoros/blob/main/kokoros-openai/src/lib.rs"
    click CoreEngine "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/Cargo.toml"
    click CoreEngine "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/lib.rs"
    click ORT_Base "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/onn/ort_base.rs"
    click ORT_Koko "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/onn/ort_koko.rs"
    click ORT_Yolo "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/onn/ort_yolo.rs"
    click Tokenize "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/tts/tokenize.rs"
    click Phonemizer "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/tts/phonemizer.rs"
    click Normalize "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/tts/normalize.rs"
    click StyleMixer "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/tts/koko.rs"
    click FileIO "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/utils/fileio.rs"
    click WAV "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/utils/wav.rs"
    click MP3 "https://github.com/lucasjinreal/kokoros/blob/main/kokoros/src/utils/mp3.rs"
    click Checkpoints "https://github.com/lucasjinreal/kokoros/tree/main/checkpoints/"
    click DataMeta "https://github.com/lucasjinreal/kokoros/tree/main/data/"
    click ScriptsDL "https://github.com/lucasjinreal/kokoros/blob/main/scripts/download_models.sh"
    click ScriptsDL "https://github.com/lucasjinreal/kokoros/blob/main/scripts/download_voices.sh"
    click ScriptsDL "https://github.com/lucasjinreal/kokoros/blob/main/scripts/requirements.txt"
    click ScriptDemo "https://github.com/lucasjinreal/kokoros/blob/main/scripts/run_openai.py"
    click Docker "https://github.com/lucasjinreal/kokoros/tree/main/Dockerfile"
    click DownloadAll "https://github.com/lucasjinreal/kokoros/blob/main/download_all.sh"
    click Install "https://github.com/lucasjinreal/kokoros/blob/main/install.sh"

    %% Styles
    classDef ui fill:#B3D4FC,stroke:#333;
    classDef core fill:#C2F0C2,stroke:#333;
    classDef backend fill:#FFD8A8,stroke:#333;
    classDef util fill:#E0E0E0,stroke:#333;
    classDef storage fill:#E8E8E8,stroke:#333;
    classDef external fill:#FFFFFF,stroke:#333;
```
