How speaches-ai/speaches works
Project Type and Purpose
• This project is an OpenAI API-compatible server that supports streaming transcription, translation, and speech generation.
• It integrates multiple capabilities:
• Speech-to-Text using faster-whisper.
• Text-to-Speech through piper and Kokoro.
• Planned extensions for audio generation and real-time interactions.
• Its purpose is to work like Ollama but for TTS/STT models, offering dynamic model loading, streaming (using SSE), and deployment via Docker/Docker Compose.
File Structure
• Top-level directories include configuration, docs, examples, scripts, src, tests, and deployment/yaml files:
• The “configuration” directory contains YAML files for observability and monitoring (Grafana, Loki, Prometheus, etc.).
• The “docs” directory and mkdocs.yml indicate that documentation is maintained with MkDocs.
• Deployment files such as Dockerfile, various docker-compose YAML files, and GitHub workflows indicate a containerized, CI/CD-ready deployment model.
• The “src/speaches” directory is the core of the application:
• It is organized into modules (api_types, audio, configuration, dependencies, etc.).
• Submodules like “realtime” (with event routers, session management, buffers) handle streaming operations.
• “routers” contain endpoints for different API functionalities (chat, misc, models, realtime, speech, stt, vad).
• There is a simple UI layer under “ui” that seems to support client applications (web UI tabs for audio chat, STT, TTS).
• The mix of configuration, API routing, real-time streaming and UI modules suggests a clear separation of concerns and layered architecture.
Key Architectural Components
• API Gateway and Routing Layer:
• The “routers” folder represents API endpoints that handle different types of requests (chat, speech, transcription, etc.).
• Core Business Logic:
• Located in “src/speaches”: including main.py for starting the server, model_manager.py for dynamic model loading/unloading, and utility modules (logger, text_utils).
• Real-time Streaming and Event Handling:
• The “realtime” folder contains many components for managing streaming data (session, event routers, pubsub, input audio buffer).
• UI Layer:
• The “ui” portion under “src/speaches/ui” for the web or client-side integration. It provides interactive components like tabs (audio_chat, stt, tts).
• Configuration & Observability:
• The “configuration” directory houses YAML files for Grafana, Prometheus, Loki, etc., indicating an emphasis on monitoring and telemetry.
• Deployment & CI/CD:
• Dockerfile, multiple docker-compose files (for CPU, CUDA, etc.), and GitHub workflows demonstrate containerized deployment and automated testing/building.
Relationships and Data Flows
• Client Applications interact with the API endpoints (routers) via HTTP/SSE for streaming responses.
• The API Gateway routes requests to the core business logic:
• For transcription: audio input is routed to the “audio” and “realtime” modules.
• For TTS: text requests are processed by the modules interfacing with TTS engines (piper, Kokoro).
• The model_manager oversees dynamic loading/unloading of models based on client requests.
• The monitoring and logging stack (configuration components) is integrated to track and observe system health.
• The UI components may interact with the routing endpoints or serve as an administrative/control interface.
Architectural Patterns & Design Principles
• Modular Structure: Clear separation of concerns (API routes, business logic, real-time processing, UI, and configuration).
• Event-driven Design: The “realtime” modules handle asynchronous events, enabling streaming and dynamic interactions.
• Containerization & Deployment Flexibility: Docker and multiple docker-compose configurations facilitate different runtime environments (CPU vs. GPU support).
• Dynamic Resource Management: Models are loaded and offloaded dynamically based on usage.