# enricollen/fastrtc-voice-agent architecture

> Architecture diagram of the GitHub repository enricollen/fastrtc-voice-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/enricollen/fastrtc-voice-agent
- Repository: https://github.com/enricollen/fastrtc-voice-agent

## Overview

FastRTC Voice Agent turns microphone audio into a transcript, runs a tool-capable conversational agent, then synthesizes and plays an audio reply. The sampled Gradio entry point shows that orchestration directly; README documents a separate custom web UI and provider options. Speech providers are selected through a shared service, while the agent uses LlamaIndex with LiteLLM and a weather tool. The custom UI wiring and several provider implementations were not sampled, so their internal connections are omitted or marked unverified.

Main entry points: [`src/speech/speech_service.py`](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/speech_service.py) (Speech service), [`src/agent.py`](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/agent.py) (Conversation agent), [`src/speech/stt/provider.py`](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/provider.py) (STT provider contract), [`main_custom_ui.py`](https://github.com/enricollen/fastrtc-voice-agent/blob/main/main_custom_ui.py) (Custom web UI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["Voice interfaces"]
  node_gradio["Gradio / phone UI<br/>[main.py]"]
  node_customui["Custom web UI<br/>[main_custom_ui.py]"]
  node_webassets["UI assets"]
end

subgraph group_conversation["Conversation and tools"]
  node_agent["Conversation agent<br/>[agent.py]"]
  node_llm["LLM service<br/>[llm_service.py]"]
  node_history["Chat history<br/>[chat_history.py]"]
  node_weather["Weather tool<br/>[weather_tool.py]"]
end

subgraph group_speech["Speech processing"]
  node_speech["Speech service<br/>[speech_service.py]"]
  node_sttbase["STT provider contract<br/>[provider.py]"]
  node_ttsbase["TTS provider contract<br/>[provider.py]"]
end

subgraph group_providers["Speech providers"]
  node_elevenstt["ElevenLabs STT<br/>[elevenlabs_stt.py]"]
  node_groqstt["Groq STT<br/>[groq_stt.py]"]
  node_openstt["OpenAI STT<br/>[openai_stt.py]"]
  node_whisper["Local Whisper STT<br/>[whisper_stt.py]"]
  node_eleventts["ElevenLabs TTS<br/>[elevenlabs_tts.py]"]
  node_kokoro["Local Kokoro TTS<br/>[kokoro_tts.py]"]
end

node_user(("Voice user"))
node_browser(("Browser client"))
node_meteo["Meteosource API"]
node_llmproviders["LLM providers"]

node_user -->|"sends audio"| node_gradio
node_gradio -->|"transcribes / synthesizes"| node_speech
node_gradio -->|"invokes"| node_agent
node_agent -->|"initializes"| node_llm
node_agent -->|"registers tool"| node_weather
node_weather -->|"requests weather"| node_meteo
node_llm -->|"queries models"| node_llmproviders
node_customui -.->|"serves interface"| node_browser
node_browser -.->|"uses assets"| node_webassets
node_customui -.->|"handles voice"| node_speech
node_customui -.->|"submits messages"| node_agent
node_speech -->|"dispatches STT"| node_elevenstt
node_speech -->|"dispatches STT"| node_groqstt
node_speech -->|"dispatches STT"| node_openstt
node_speech -->|"dispatches STT"| node_whisper
node_speech -->|"dispatches TTS"| node_eleventts
node_speech -->|"dispatches TTS"| node_kokoro
node_elevenstt -->|"implements"| node_sttbase
node_groqstt -->|"implements"| node_sttbase
node_openstt -->|"implements"| node_sttbase
node_whisper -->|"implements"| node_sttbase
node_eleventts -->|"implements"| node_ttsbase
node_kokoro -->|"implements"| node_ttsbase

click node_gradio "https://github.com/enricollen/fastrtc-voice-agent/blob/main/main.py"
click node_customui "https://github.com/enricollen/fastrtc-voice-agent/blob/main/main_custom_ui.py"
click node_webassets "https://github.com/enricollen/fastrtc-voice-agent/tree/main/static"
click node_agent "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/agent.py"
click node_llm "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/llm_service.py"
click node_history "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/chat_history.py"
click node_weather "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/tools/weather_tool.py"
click node_speech "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/speech_service.py"
click node_sttbase "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/provider.py"
click node_ttsbase "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/tts/provider.py"
click node_elevenstt "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/elevenlabs_stt.py"
click node_groqstt "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/groq_stt.py"
click node_openstt "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/openai_stt.py"
click node_whisper "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/whisper_stt.py"
click node_eleventts "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/tts/elevenlabs_tts.py"
click node_kokoro "https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/tts/kokoro_tts.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_gradio,node_customui,node_webassets,node_user,node_browser toneBlue
class node_agent,node_llm,node_history,node_weather toneAmber
class node_speech,node_sttbase,node_ttsbase,node_meteo toneMint
class node_elevenstt,node_groqstt,node_openstt,node_whisper,node_eleventts,node_kokoro toneRose
class node_llmproviders toneIndigo
```

## Components

### Voice interfaces

- [Gradio / phone UI](https://github.com/enricollen/fastrtc-voice-agent/blob/main/main.py): `main.py`
- [Custom web UI](https://github.com/enricollen/fastrtc-voice-agent/blob/main/main_custom_ui.py): `main_custom_ui.py`
- [UI assets](https://github.com/enricollen/fastrtc-voice-agent/tree/main/static): `static`

### Conversation and tools

- [Conversation agent](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/agent.py): `src/agent.py`
- [LLM service](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/llm_service.py): `src/llm_service.py`
- [Chat history](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/chat_history.py): `src/chat_history.py`
- [Weather tool](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/tools/weather_tool.py): `src/tools/weather_tool.py`

### Speech processing

- [Speech service](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/speech_service.py): `src/speech/speech_service.py`
- [STT provider contract](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/provider.py): `src/speech/stt/provider.py`
- [TTS provider contract](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/tts/provider.py): `src/speech/tts/provider.py`

### Speech providers

- [ElevenLabs STT](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/elevenlabs_stt.py): `src/speech/stt/elevenlabs_stt.py`
- [Groq STT](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/groq_stt.py): `src/speech/stt/groq_stt.py`
- [OpenAI STT](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/openai_stt.py): `src/speech/stt/openai_stt.py`
- [Local Whisper STT](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/stt/whisper_stt.py): `src/speech/stt/whisper_stt.py`
- [ElevenLabs TTS](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/tts/elevenlabs_tts.py): `src/speech/tts/elevenlabs_tts.py`
- [Local Kokoro TTS](https://github.com/enricollen/fastrtc-voice-agent/blob/main/src/speech/tts/kokoro_tts.py): `src/speech/tts/kokoro_tts.py`

### Other components

- **Voice user**
- **Browser client**
- **Meteosource API**
- **LLM providers**

## Connections

- Voice user → Gradio / phone UI: sends audio (evidence: `main.py`)
- Gradio / phone UI → Speech service: transcribes / synthesizes (evidence: `main.py`)
- Gradio / phone UI → Conversation agent: invokes (evidence: `main.py`)
- Conversation agent → LLM service: initializes (evidence: `src/agent.py`)
- Conversation agent → Weather tool: registers tool (evidence: `src/agent.py`)
- Weather tool → Meteosource API: requests weather (evidence: `src/tools/weather_tool.py`)
- LLM service → LLM providers: queries models (evidence: `README.md`)
- Custom web UI → Browser client: serves interface (evidence: `README.md`)
- Browser client → UI assets: uses assets
- Custom web UI → Speech service: handles voice
- Custom web UI → Conversation agent: submits messages (evidence: `main_custom_ui.py`)
- Speech service → ElevenLabs STT: dispatches STT (evidence: `src/speech/speech_service.py`)
- Speech service → Groq STT: dispatches STT (evidence: `src/speech/speech_service.py`)
- Speech service → OpenAI STT: dispatches STT (evidence: `src/speech/speech_service.py`)
- Speech service → Local Whisper STT: dispatches STT (evidence: `src/speech/speech_service.py`)
- Speech service → ElevenLabs TTS: dispatches TTS (evidence: `src/speech/speech_service.py`)
- Speech service → Local Kokoro TTS: dispatches TTS (evidence: `src/speech/speech_service.py`)
- ElevenLabs STT → STT provider contract: implements (evidence: `src/speech/stt/elevenlabs_stt.py`)
- Groq STT → STT provider contract: implements (evidence: `src/speech/stt/groq_stt.py`)
- OpenAI STT → STT provider contract: implements (evidence: `src/speech/stt/openai_stt.py`)
- Local Whisper STT → STT provider contract: implements (evidence: `src/speech/stt/whisper_stt.py`)
- ElevenLabs TTS → TTS provider contract: implements (evidence: `src/speech/tts/elevenlabs_tts.py`)
- Local Kokoro TTS → TTS provider contract: implements (evidence: `src/speech/tts/kokoro_tts.py`)