# koljab/realtimetts architecture

> Architecture diagram of the GitHub repository koljab/realtimetts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/koljab/realtimetts
- Repository: https://github.com/koljab/realtimetts

## Overview

##### Project Type and Purpose
- This is a specialized real‑time text-to‑speech library that converts incoming text streams into audio in near‑real‑time. It supports multiple TTS engines including local ones (e.g., SystemEngine, CoquiEngine, PiperEngine) and remote services (e.g., Azure, Elevenlabs, OpenAI). It’s designed for low latency and high-quality audio synthesis, with fallback mechanisms when an engine fails.
- The README emphasizes integration with external APIs as well as local synthesizers, pointing out its modular and extensible nature. The project also includes examples (e.g., FastAPI integration and dockerized deployments) and extensive tests.

##### File Structure
- The main library is under the top‑level “RealtimeTTS” directory, which includes:
- Core modules: __init__.py, stream_player.py, text_to_stream.py, threadsafe_generators.py – these are likely responsible for streaming and audio playback functionality.
- An “engines” subdirectory contains multiple engine implementations (e.g., azure_engine.py, coqui_engine.py, edge_engine.py, openai_engine.py, etc.) along with JSON configuration files (e.g., male.json, female.json) that configure voice parameters.
- Other directories include:
- “docker” – contains Dockerfile and a sample application, indicating containerized deployments.
- “docs” – documentation in different languages.
- “example_fast_api” and “example_rvc” – sample integrations and use-case demos.
- “tests” – unit and integration tests for various engine configurations.
- Additional build/configuration files (e.g., setup.py, requirements.txt, .gitignore) define dependencies and packaging.

##### Main Components
a. Core Synthesis Pipeline:
- Text Input Module: This would represent how text is fed into the system (strings, generators, iterators) and then handled by the TextToAudioStream class.
- Sentence Tokenizer: Show the integration with tokenizer libraries (“nltk” or “stanza”) used to split text into sentences.
- Audio Buffer & Playback Controller: Represent modules like stream_player.py and handling of realtime audio streaming and buffering.

b. TTS Engine Abstraction:
- BaseEngine: Depicts the abstraction that all engine implementations inherit from.
- Fallback Mechanism: Show that a list of engine instances can be provided and the system will automatically switch engines in case of errors or delays.

c. External Integration & Dependencies:

d. Additional Support Modules:
- Thread-Safe Generators: Represent modules that ensure asynchronous data feeding for real-time voice synthesis.
- Callback Hooks: Illustrate the event/callback range (on_text_stream_start, on_audio_stream_start, on_sentence_synthesized, etc.) that aid in interfacing with UIs or other front-end elements.
- Configuration & Dependency Modules: Optionally, add notes on configuration files/settings (JSON files in the engines folder) and environment variable management (e.g., API keys).

e. Example and Deployment Layers:
- Application Integration Examples: Represent “example_fast_api” and “docker” directories as separate integration nodes showing how the library can be embedded in a larger system.
- Tests: You might show a “Testing” component that verifies the TTS functionalities using various engine test scripts, ensuring reliability.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Synthesis Pipeline
    subgraph "Core Synthesis Pipeline"
        TextInput["Text Input Module"] 
        SentenceTokenizer["Sentence Tokenizer"]
        ThreadSafe["Thread-Safe Generators"]
        TextStream["TextToAudioStream"]
        AudioBuffer["Audio Buffer & Playback Controller"]
        
        TextInput -->|"processes"| SentenceTokenizer
        SentenceTokenizer -->|"feeds"| TextStream
        ThreadSafe -->|"supports"| TextStream
        TextStream -->|"outputs to"| AudioBuffer
    end

    %% TTS Engine Processing
    subgraph "TTS Engine Processing"
        BaseEngine["BaseEngine"]
        Fallback["Fallback Mechanism"]
        
        TextStream -->|"synthesizes via"| BaseEngine
        BaseEngine -->|"manages"| Fallback

        subgraph "Engine Implementations"
            Azure["AzureEngine"]
            Coqui["CoquiEngine"]
            Edge["EdgeEngine"]
            Elevenlabs["ElevenlabsEngine"]
            GTTS["GTTS Engine"]
            Kokoro["KokoroEngine"]
            OpenAI["OpenAIEngine"]
            Orpheus["OrpheusEngine"]
            Parler["ParlerEngine"]
            Piper["PiperEngine"]
            Style["StyleEngine"]
            System["SystemEngine"]
        end

        Fallback ---|"fallback to"| Azure
        Fallback ---|"fallback to"| Coqui
        Fallback ---|"fallback to"| Edge
        Fallback ---|"fallback to"| Elevenlabs
        Fallback ---|"fallback to"| GTTS
        Fallback ---|"fallback to"| Kokoro
        Fallback ---|"fallback to"| OpenAI
        Fallback ---|"fallback to"| Orpheus
        Fallback ---|"fallback to"| Parler
        Fallback ---|"fallback to"| Piper
        Fallback ---|"fallback to"| Style
        Fallback ---|"fallback to"| System
    end

    %% External Integrations and Dependencies
    subgraph "External Integrations"
        AzureAPI["Microsoft Azure API"]
        ElevenlabsAPI["Elevenlabs API"]
        OpenAIApi["OpenAI API"]
        EngineConfig["Engine Config Files"]
        DockerExample["Docker Deployment Example"]
        FastAPIExample["FastAPI Integration"]
        Tests["Tests"]
    end

    %% Connections to External Services
    AzureAPI ---|"requires"| Azure
    ElevenlabsAPI ---|"requires"| Elevenlabs
    OpenAIApi ---|"requires"| OpenAI
    BaseEngine ---|"configured by"| EngineConfig

    %% Integration Connections
    AudioBuffer -->|"integrated with"| FastAPIExample
    AudioBuffer -->|"containerized by"| DockerExample
    AudioBuffer -->|"verified by"| Tests

    %% Class Definitions for Styling
    class TextInput,SentenceTokenizer,TextStream,AudioBuffer,ThreadSafe core
    class BaseEngine,Fallback core
    class Azure,Coqui,Edge,Elevenlabs,GTTS,Kokoro,OpenAI,Orpheus,Parler,Piper,Style,System engine
    class AzureAPI,ElevenlabsAPI,OpenAIApi,EngineConfig external
    class DockerExample,FastAPIExample,Tests integration

    %% Styles
    classDef core fill:#bbdefb,stroke:#0d47a1,stroke-width:2px;
    classDef engine fill:#ffe0b2,stroke:#e65100,stroke-width:2px;
    classDef external fill:#c8e6c9,stroke:#1b5e20,stroke-width:2px;
    classDef integration fill:#fff9c4,stroke:#f57f17,stroke-dasharray: 5 5;

    %% Click Events
    click TextInput "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/text_to_stream.py"
    click AudioBuffer "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/stream_player.py"
    click ThreadSafe "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/threadsafe_generators.py"
    click BaseEngine "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/base_engine.py"
    click Azure "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/azure_engine.py"
    click Coqui "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/coqui_engine.py"
    click Edge "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/edge_engine.py"
    click Elevenlabs "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/elevenlabs_engine.py"
    click GTTS "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/gtts_engine.py"
    click Kokoro "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/kokoro_engine.py"
    click OpenAI "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/openai_engine.py"
    click Orpheus "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/orpheus_engine.py"
    click Parler "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/parler_engine.py"
    click Piper "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/piper_engine.py"
    click Style "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/style_engine.py"
    click System "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/system_engine.py"
    click EngineConfig "https://github.com/koljab/realtimetts/blob/master/RealtimeTTS/engines/male.json"
    click DockerExample "https://github.com/koljab/realtimetts/tree/master/docker/openai-tts-docker/"
    click FastAPIExample "https://github.com/koljab/realtimetts/tree/master/example_fast_api/"
    click Tests "https://github.com/koljab/realtimetts/tree/master/tests/"
```
