# speaches-ai/speaches architecture

> Architecture diagram of the GitHub repository speaches-ai/speaches, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/speaches-ai/speaches
- Repository: https://github.com/speaches-ai/speaches

## Overview

##### Project Type and Purpose
- This project is an OpenAI API-compatible server that supports streaming transcription, translation, and speech generation.
- It integrates multiple capabilities:
- Speech-to-Text using faster-whisper.
- Text-to-Speech through piper and Kokoro.
- Planned extensions for audio generation and real-time interactions.
- Its purpose is to work like Ollama but for TTS/STT models, offering dynamic model loading, streaming (using SSE), and deployment via Docker/Docker Compose.

##### File Structure
- Top-level directories include configuration, docs, examples, scripts, src, tests, and deployment/yaml files:
- The “configuration” directory contains YAML files for observability and monitoring (Grafana, Loki, Prometheus, etc.).
- The “docs” directory and mkdocs.yml indicate that documentation is maintained with MkDocs.
- Deployment files such as Dockerfile, various docker-compose YAML files, and GitHub workflows indicate a containerized, CI/CD-ready deployment model.
- The “src/speaches” directory is the core of the application:
- It is organized into modules (api_types, audio, configuration, dependencies, etc.).
- Submodules like “realtime” (with event routers, session management, buffers) handle streaming operations.
- “routers” contain endpoints for different API functionalities (chat, misc, models, realtime, speech, stt, vad).
- There is a simple UI layer under “ui” that seems to support client applications (web UI tabs for audio chat, STT, TTS).
- The mix of configuration, API routing, real-time streaming and UI modules suggests a clear separation of concerns and layered architecture.

##### Key Architectural Components
- API Gateway and Routing Layer:
- The “routers” folder represents API endpoints that handle different types of requests (chat, speech, transcription, etc.).
- Core Business Logic:
- Located in “src/speaches”: including main.py for starting the server, model_manager.py for dynamic model loading/unloading, and utility modules (logger, text_utils).
- Real-time Streaming and Event Handling:
- The “realtime” folder contains many components for managing streaming data (session, event routers, pubsub, input audio buffer).
- UI Layer:
- The “ui” portion under “src/speaches/ui” for the web or client-side integration. It provides interactive components like tabs (audio_chat, stt, tts).
- Configuration & Observability:
- The “configuration” directory houses YAML files for Grafana, Prometheus, Loki, etc., indicating an emphasis on monitoring and telemetry.
- Deployment & CI/CD:
- Dockerfile, multiple docker-compose files (for CPU, CUDA, etc.), and GitHub workflows demonstrate containerized deployment and automated testing/building.

##### Relationships and Data Flows
- Client Applications interact with the API endpoints (routers) via HTTP/SSE for streaming responses.
- The API Gateway routes requests to the core business logic:
- For transcription: audio input is routed to the “audio” and “realtime” modules.
- For TTS: text requests are processed by the modules interfacing with TTS engines (piper, Kokoro).
- The model_manager oversees dynamic loading/unloading of models based on client requests.
- The monitoring and logging stack (configuration components) is integrated to track and observe system health.
- The UI components may interact with the routing endpoints or serve as an administrative/control interface.

##### Architectural Patterns & Design Principles
- Modular Structure: Clear separation of concerns (API routes, business logic, real-time processing, UI, and configuration).
- Event-driven Design: The “realtime” modules handle asynchronous events, enabling streaming and dynamic interactions.
- Containerization & Deployment Flexibility: Docker and multiple docker-compose configurations facilitate different runtime environments (CPU vs. GPU support).
- Dynamic Resource Management: Models are loaded and offloaded dynamically based on usage.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        Client["Client (Web UI / API Consumer)"]:::client
    end

    %% API Gateway & Routing Layer
    subgraph "API Gateway & Routing Layer"
        APIGateway["API Gateway & Routing Layer"]:::api
        ChatRouter["Chat Router"]:::api
        MiscRouter["Misc Router"]:::api
        ModelsRouter["Models Router"]:::api
        RealtimeRouter["Realtime Router"]:::api
        SpeechRouter["Speech Router"]:::api
        STTRouter["STT Router"]:::api
        VADRouter["VAD Router"]:::api
        APIGateway --> ChatRouter
        APIGateway --> MiscRouter
        APIGateway --> ModelsRouter
        APIGateway --> RealtimeRouter
        APIGateway --> SpeechRouter
        APIGateway --> STTRouter
        APIGateway --> VADRouter
    end

    %% Core Business Logic
    subgraph "Core Business Logic"
        Core["Core Business Logic"]:::core
        Main["main.py"]:::core
        ModelManager["Model Manager"]:::core
        Utils["Text Utils, Logger & API Types"]:::core
        Core --> Main
        Core --> ModelManager
        Core --> Utils
    end

    %% Real-time Streaming and Event Handling
    subgraph "Real-time Streaming & Event Handling"
        RealtimeModule["Realtime Module"]:::realtime
        SessionManager["Session Manager"]:::realtime
        EventRouter["Event Router"]:::realtime
        ConvEventRouter["Conversation Event Router"]:::realtime
        AudioBuffer["Input Audio Buffer"]:::realtime
        PubSub["PubSub"]:::realtime
        RealtimeModule --> SessionManager
        RealtimeModule --> EventRouter
        RealtimeModule --> ConvEventRouter
        RealtimeModule --> AudioBuffer
        RealtimeModule --> PubSub
    end

    %% UI Layer
    subgraph "UI Layer"
        UILayer["UI Layer"]:::ui
        AppFile["app.py"]:::ui
        AudioChatTab["Audio Chat Tab"]:::ui
        STTTab["STT Tab"]:::ui
        TTSTab["TTS Tab"]:::ui
        UILayer --> AppFile
        UILayer --> AudioChatTab
        UILayer --> STTTab
        UILayer --> TTSTab
    end

    %% External Integrations
    subgraph "External Integrations"
        TTSEngines["TTS Engines (Piper, Kokoro)"]:::external
        STTEngine["STT Engine (faster-whisper)"]:::external
    end

    %% Monitoring & Observability
    subgraph "Monitoring & Observability"
        Monitoring["Monitoring Services (Grafana, Prometheus, Loki, OTel)"]:::monitoring
    end

    %% Deployment & CI/CD
    subgraph "Deployment & CI/CD Infrastructure"
        Deployment["Deployment & CI/CD Infrastructure"]:::deployment
    end

    %% Relationships and Data Flow
    Client -->|"SendsRequests"| APIGateway
    APIGateway -->|"RoutesTo"| Core
    Core -->|"Triggers"| RealtimeModule
    Core -->|"Utilizes"| TTSEngines
    Core -->|"Utilizes"| STTEngine
    Core -->|"SendsLogsTo"| Monitoring
    UILayer -->|"InteractsWith"| APIGateway
    Deployment ---|"Hosts"| Core
    Deployment ---|"Manages"| APIGateway
    Deployment ---|"Deploys"| UILayer

    %% Click Events (do not display URLs)
    click APIGateway "https://github.com/speaches-ai/speaches/tree/master/src/speaches/routers"
    click Core "https://github.com/speaches-ai/speaches/tree/master/src/speaches"
    click RealtimeModule "https://github.com/speaches-ai/speaches/tree/master/src/speaches/realtime"
    click UILayer "https://github.com/speaches-ai/speaches/tree/master/src/speaches/ui"
    click Monitoring "https://github.com/speaches-ai/speaches/tree/master/configuration"
    click Deployment "https://github.com/speaches-ai/speaches/tree/master/Dockerfile"

    %% Styles
    classDef client fill:#ffcccc,stroke:#333,stroke-width:2px;
    classDef api fill:#cce5ff,stroke:#333,stroke-width:2px;
    classDef core fill:#d4edda,stroke:#333,stroke-width:2px;
    classDef realtime fill:#fff3cd,stroke:#333,stroke-width:2px;
    classDef ui fill:#e2e3e5,stroke:#333,stroke-width:2px;
    classDef external fill:#f8d7da,stroke:#333,stroke-width:2px;
    classDef monitoring fill:#d1ecf1,stroke:#333,stroke-width:2px;
    classDef deployment fill:#fefefe,stroke:#333,stroke-width:2px;
```
