# rimelabs/rime-livekit-agents architecture

> Architecture diagram of the GitHub repository rimelabs/rime-livekit-agents, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rimelabs/rime-livekit-agents
- Repository: https://github.com/rimelabs/rime-livekit-agents

## Overview

##### Project type and purpose
- This is a backend “voice agent” application written in Python. It connects to a LiveKit media server, listens for user audio, processes text (via prompts and utilities), renders TTS calls to Rime.ai models, and sends synthesized audio back over LiveKit. There is no frontend UI in this repo—this is an autonomous agent/service.

##### File structure
- Top-level scripts and modules:
- rime_agent.py – main entrypoint, CLI commands (console mode, download-files, LiveKit loop)
- agent_configs.py – definitions of voice configurations & prompt templates (VOICE)
- text_utils.py – text normalization, turn-detection model loader, utility routines
- Project config:
- .env.example for environment variables (OpenAI, Rime, LiveKit credentials)
- requirements.txt pins dependencies including a patched livekit-plugins-rime
- There are no separate “frontend” or “database” directories—this is a single service binary with external API integrations.

##### Key insights from the README
- Dependencies on external services: LiveKit server, Rime.ai TTS, OpenAI API
- CLI modes: console for local testing vs. LiveKit mode for production
- Configurable via environment and agent_configs.py
- Optional next steps suggest a standalone frontend could be added, but is outside scope here

##### Main components & interactions
a. Internal components (all within the Python agent process):
1. Configuration Loader
2. Agent Core / Event Loop
3. LiveKit Connector (SDK client)
4. Audio I/O Handler (capture/playback)
5. Text Utils (sentence segmentation, turn detection)
6. Prompt Manager (reads templates from agent_configs.py)
7. TTS Client (invokes Rime.ai `mistv2` or `arcana` models)
8. (Optional) OpenAI Client (if prompts use LLM completions)
9. Download Manager (fetches model files for turn detection)
b. External components:
- LiveKit Server (media router)
- Rime.ai TTS Service
- OpenAI API
c. Data flows / relationships:
1. LiveKit Server streams user audio → LiveKit Connector → Audio I/O Handler
2. Audio → Text Utils (detect speech boundaries) → Prompt Manager
3. Prompt text → (optionally OpenAI) → finalized prompt → TTS Client
4. TTS Client sends HTTP to Rime.ai → receives audio blob
5. Audio blob → Audio I/O Handler → LiveKit Connector → LiveKit Server → user
6. Configuration Loader feeds all modules at startup
7. Download Manager invoked at CLI “download-files” to pull models locally

##### Architectural patterns & design principles
- Modular separation: config / utils / core logic / external connectors
- Single-process, event-driven service (no persistent database)
- Clear dependency inversion: TTS and OpenAI behind client adapters
- CLI commands for lifecycle management (download vs. run)

##### Detail
- How environment variables flow into the Configuration Loader.
- Make the audio/text flow explicit at each step.
- Don’t over-simplify: include the Download Manager path and separate the OpenAI client even if not used immediately in voice-only mode.

## Diagram (Mermaid)

```mermaid
graph TD
    %% External User and Services
    User(("User (Audio)")):::external
    LiveKitServer((LiveKit Server)):::external
    RimeService((Rime.ai TTS Service)):::external
    OpenAIAPI((OpenAI API)):::external

    %% Configuration and CLI
    ConfigLoader["Configuration Loader\n(.env.example)"]:::config
    DownloadManager["Download Manager\n(download-files CLI)"]:::internal

    %% Agent Service Modules
    subgraph "Rime Agent Service"
        AgentCore["Agent Core / Event Loop"]:::internal
        LiveKitConnector["LiveKit Connector\n(SDK client)"]:::internal
        AudioIO["Audio I/O Handler"]:::internal
        TextUtils["Text Utils\n(text_utils.py)"]:::internal
        PromptManager["Prompt Manager\n(agent_configs.py)"]:::internal
        TTSClient["TTS Client\n(Rime.ai HTTP)"]:::internal
        OpenAIClient["OpenAI Client"]:::internal
    end

    %% Data Flows
    User --> LiveKitServer
    LiveKitServer --> LiveKitConnector
    LiveKitConnector --> AudioIO
    AudioIO --> TextUtils
    TextUtils --> PromptManager
    PromptManager -->|optional prompt completion| OpenAIClient
    OpenAIClient --> PromptManager
    PromptManager --> TTSClient
    TTSClient -->|HTTP audio blob| AudioIO
    AudioIO --> LiveKitConnector
    LiveKitConnector --> LiveKitServer
    LiveKitServer --> User

    %% Configuration Flows
    ConfigLoader --> AgentCore
    ConfigLoader --> LiveKitConnector
    ConfigLoader --> AudioIO
    ConfigLoader --> TextUtils
    ConfigLoader --> PromptManager
    ConfigLoader --> TTSClient
    ConfigLoader --> DownloadManager

    %% Download Manager Operation
    DownloadManager --> TextUtils

    %% Click Events
    click ConfigLoader "https://github.com/rimelabs/rime-livekit-agents/blob/main/.env.example"
    click AgentCore "https://github.com/rimelabs/rime-livekit-agents/blob/main/rime_agent.py"
    click LiveKitConnector "https://github.com/rimelabs/rime-livekit-agents/blob/main/rime_agent.py"
    click AudioIO "https://github.com/rimelabs/rime-livekit-agents/blob/main/rime_agent.py"
    click TextUtils "https://github.com/rimelabs/rime-livekit-agents/blob/main/text_utils.py"
    click PromptManager "https://github.com/rimelabs/rime-livekit-agents/blob/main/agent_configs.py"
    click TTSClient "https://github.com/rimelabs/rime-livekit-agents/blob/main/rime_agent.py"
    click OpenAIClient "https://github.com/rimelabs/rime-livekit-agents/blob/main/text_utils.py"
    click DownloadManager "https://github.com/rimelabs/rime-livekit-agents/blob/main/rime_agent.py"

    %% Styles
    classDef internal fill:#ADD8E6,stroke:#000;
    classDef external fill:#90EE90,stroke:#000;
    classDef config fill:#D3D3D3,stroke:#000;
```
