# remsky/kokoro-fastapi architecture

> Architecture diagram of the GitHub repository remsky/kokoro-fastapi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/remsky/kokoro-fastapi
- Repository: https://github.com/remsky/kokoro-fastapi

## Overview

Based on our review of the file tree and README content, this project is a Dockerized FastAPI wrapper offering text-to-speech services by interfacing with the Kokoro-82M model (and various voice packs) via a REST API. It supports both CPU and NVIDIA GPU inference with multiple endpoints (including OpenAI-compatible speech, streaming audio, phoneme processing, debugging endpoints, etc.) and provides a web UI alongside a standalone web client.

##### Main Components
- API Server: The FastAPI backend (located under the “api” directory) provides endpoints for audio speech generation, phoneme processing, voice mixing, and debugging.
- Inference & Processing Layers: Under “api/src/inference” you have inference managers (e.g., model_manager.py, voice_manager.py) that handle TTS synthesis (which in turn rely on PyTorch, CUDA for GPU acceleration, or CPU inference).
- Voice Resources: The “api/src/voices” directory holds multiple model files (.pt files) for the various voice packs used during synthesis.
- User Interface: The “ui” and “web” directories represent separate UI layers – one being the FastAPI-driven UI and the other a web (likely React-based) client that interacts with the API.
- Docker & Deployment: The “docker”, “charts”, and GitHub workflow directories indicate containerized deployment options (Docker Compose, Helm charts for Kubernetes, etc.). Include these to show deployment orchestration.
- External Integrations: Highlight the compatibility with OpenAI’s API interface and the integration for streaming support via services like PyAudio (as seen in examples).

##### Relationships and Interactions
- Clients (via web UI, API calls, or direct script examples) send HTTP requests (REST) to the FastAPI server.
- The API routes (in “api/src/routers”) dispatch these requests to core service modules.
- The core and inference layers process the input text by converting it to phonemes, splitting tokens, and managing model configurations.
- The inference manager interacts with the voice models (from “api/src/voices”) to generate audio output.
- Debug endpoints and streaming services provide performance metrics and real-time responses.
- For deployments, container orchestration (with separate Docker files for CPU and GPU) manages the environment, and CI/CD pipelines (GitHub workflows) enable build automation.

##### Architectural Patterns & Design Principles
- Layered Architecture: Segregate presentation (API/UI), business logic (services, inference), and data (models, voice files).
- Containerization & Infrastructure as Code: With defined Helm charts and Docker Compose scripts, the deployment environment is well encapsulated.
- Extensibility: The modular design (e.g., separate endpoints for phoneme generation and debug, clear separation of text processing) ensures that additional features or integrations (such as ONNX support) can be introduced.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Clients Layer
    subgraph "Clients Layer"
        A("Clients/External Requests"):::clients
    end

    %% API Layer
    subgraph "API Layer"
        B("FastAPI Server"):::api
        C("API Routers (OpenAI-Compatible)"):::external
        D("Streaming Audio Writer"):::external
    end

    %% Core & Inference Layer
    subgraph "Core & Inference Layer"
        E("Core Services & Text Processing"):::core
        F("Model Configurations"):::core
        G("Inference Manager & Voice Manager"):::inference
        H("Voice Resources"):::inference
    end

    %% User Interface Layer
    subgraph "User Interface Layer"
        I("FastAPI-driven UI"):::ui
        J("Web Client"):::ui
    end

    %% Deployment Layer
    subgraph "Deployment Layer"
        K("Docker (CPU/GPU)"):::deploy
        L("Kubernetes/Helm Charts"):::deploy
        M("CI/CD Pipelines"):::deploy
    end

    %% Data Flow Connections
    A --> B
    B --> C
    C --> D
    B --> E
    E --> F
    E --> G
    G --> H
    A --> I
    A --> J
    B --> K
    K --> L
    K --> M

    %% Click Events
    click B "https://github.com/remsky/kokoro-fastapi/blob/master/api/src/main.py"
    click G "https://github.com/remsky/kokoro-fastapi/tree/master/api/src/inference"
    click E "https://github.com/remsky/kokoro-fastapi/tree/master/api/src/core"
    click F "https://github.com/remsky/kokoro-fastapi/tree/master/api/src/models"
    click H "https://github.com/remsky/kokoro-fastapi/tree/master/api/src/voices"
    click I "https://github.com/remsky/kokoro-fastapi/tree/master/ui/"
    click J "https://github.com/remsky/kokoro-fastapi/tree/master/web/"
    click K "https://github.com/remsky/kokoro-fastapi/tree/master/docker/"
    click L "https://github.com/remsky/kokoro-fastapi/tree/master/charts/kokoro-fastapi"
    click M "https://github.com/remsky/kokoro-fastapi/tree/master/.github/workflows"
    click C "https://github.com/remsky/kokoro-fastapi/blob/master/api/src/routers/openai_compatible.py"
    click D "https://github.com/remsky/kokoro-fastapi/blob/master/api/src/services/streaming_audio_writer.py"

    %% Styles
    classDef clients fill:#CCE5FF,stroke:#2B82D7,stroke-width:2px;
    classDef api fill:#BBDEFB,stroke:#1E88E5,stroke-width:2px;
    classDef core fill:#C8E6C9,stroke:#43A047,stroke-width:2px;
    classDef inference fill:#FFF9C4,stroke:#FBC02D,stroke-width:2px;
    classDef ui fill:#F8BBD0,stroke:#D81B60,stroke-width:2px;
    classDef deploy fill:#D1C4E9,stroke:#673AB7,stroke-width:2px;
    classDef external fill:#FFCCBC,stroke:#FB8C00,stroke-width:2px;
```
