How remsky/kokoro-fastapi works
Based on our review of the file tree and README content, this project is a Dockerized FastAPI wrapper offering text-to-speech services by interfacing with the Kokoro-82M model (and various voice packs) via a REST API. It supports both CPU and NVIDIA GPU inference with multiple endpoints (including OpenAI-compatible speech, streaming audio, phoneme processing, debugging endpoints, etc.) and provides a web UI alongside a standalone web client.
Main Components
• API Server: The FastAPI backend (located under the “api” directory) provides endpoints for audio speech generation, phoneme processing, voice mixing, and debugging.
• Inference & Processing Layers: Under “api/src/inference” you have inference managers (e.g., model_manager.py, voice_manager.py) that handle TTS synthesis (which in turn rely on PyTorch, CUDA for GPU acceleration, or CPU inference).
• Voice Resources: The “api/src/voices” directory holds multiple model files (.pt files) for the various voice packs used during synthesis.
• User Interface: The “ui” and “web” directories represent separate UI layers – one being the FastAPI-driven UI and the other a web (likely React-based) client that interacts with the API.
• Docker & Deployment: The “docker”, “charts”, and GitHub workflow directories indicate containerized deployment options (Docker Compose, Helm charts for Kubernetes, etc.). Include these to show deployment orchestration.
• External Integrations: Highlight the compatibility with OpenAI’s API interface and the integration for streaming support via services like PyAudio (as seen in examples).
Relationships and Interactions
• Clients (via web UI, API calls, or direct script examples) send HTTP requests (REST) to the FastAPI server.
• The API routes (in “api/src/routers”) dispatch these requests to core service modules.
• The core and inference layers process the input text by converting it to phonemes, splitting tokens, and managing model configurations.
• The inference manager interacts with the voice models (from “api/src/voices”) to generate audio output.
• Debug endpoints and streaming services provide performance metrics and real-time responses.
• For deployments, container orchestration (with separate Docker files for CPU and GPU) manages the environment, and CI/CD pipelines (GitHub workflows) enable build automation.
Architectural Patterns & Design Principles
• Layered Architecture: Segregate presentation (API/UI), business logic (services, inference), and data (models, voice files).
• Containerization & Infrastructure as Code: With defined Helm charts and Docker Compose scripts, the deployment environment is well encapsulated.
• Extensibility: The modular design (e.g., separate endpoints for phoneme generation and debug, clear separation of text processing) ensures that additional features or integrations (such as ONNX support) can be introduced.