# sankar-s-r/voice_agent architecture

> Architecture diagram of the GitHub repository sankar-s-r/voice_agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sankar-s-r/voice_agent
- Repository: https://github.com/sankar-s-r/voice_agent

## Overview

Voice Agent is a real-time, streaming voice assistant. A user speaks over WebRTC; Pipecat’s pipeline performs turn detection, transcription, LLM response generation, and speech synthesis before returning audio to the user. A separate observer monitors turn and speech events and injects a natural filler if the response is delayed. The README documents the service choices and streaming design; source excerpts show pipeline wiring and fallback behavior, but do not expose the internals of Pipecat or the external providers.

Main entry points: [`server/bot.py`](https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py) (WebRTC Transport), [`server/fallback.py`](https://github.com/sankar-s-r/voice_agent/blob/main/server/fallback.py) (Fallback Observer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_conversation["Voice Conversation"]
  node_webrtc["WebRTC Transport<br/>[bot.py]"]
  node_turn["Turn Detection<br/>[bot.py]"]
  node_stt["Streaming STT<br/>[bot.py]"]
  node_context["Conversation Context<br/>[bot.py]"]
  node_llm["Streaming LLM<br/>[bot.py]"]
  node_tts["Streaming TTS<br/>[bot.py]"]
  node_speaker(("Audio Output"))
end

subgraph group_orchestration["Pipeline Orchestration"]
  node_worker["Pipeline Worker<br/>[bot.py]"]
  node_runner["Pipeline Runner<br/>[bot.py]"]
end

subgraph group_engagement["Fallback Engagement"]
  node_fallback["Fallback Observer<br/>[fallback.py]"]
end

subgraph group_providers["External Providers"]
  node_deepgram["Deepgram"]
  node_groq["Groq"]
  node_cartesia["Cartesia"]
end

node_user(("Voice User"))

node_user -->|"sends audio"| node_webrtc
node_webrtc -->|"streams input"| node_stt
node_stt -->|"transcribes"| node_turn
node_turn -->|"forwards turn"| node_llm
node_context -->|"provides context"| node_llm
node_llm -->|"streams response"| node_tts
node_tts -->|"sends audio"| node_webrtc
node_webrtc -->|"plays audio"| node_speaker
node_worker -->|"runs pipeline"| node_webrtc
node_runner -->|"runs worker"| node_worker
node_worker -->|"notifies observer"| node_fallback
node_fallback -->|"queues filler"| node_worker
node_stt -->|"transcribes with"| node_deepgram
node_llm -->|"generates with"| node_groq
node_tts -->|"synthesizes with"| node_cartesia

click node_webrtc "https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py"
click node_turn "https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py"
click node_stt "https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py"
click node_context "https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py"
click node_llm "https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py"
click node_tts "https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py"
click node_worker "https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py"
click node_runner "https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py"
click node_fallback "https://github.com/sankar-s-r/voice_agent/blob/main/server/fallback.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_webrtc,node_turn,node_stt,node_context,node_llm,node_tts,node_speaker,node_user toneBlue
class node_worker,node_runner toneAmber
class node_fallback toneMint
class node_deepgram,node_groq,node_cartesia toneRose
```

## Components

### Voice Conversation

- [WebRTC Transport](https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py): `server/bot.py`
- [Turn Detection](https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py): `server/bot.py`
- [Streaming STT](https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py): `server/bot.py`
- [Conversation Context](https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py): `server/bot.py`
- [Streaming LLM](https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py): `server/bot.py`
- [Streaming TTS](https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py): `server/bot.py`
- **Audio Output**

### Pipeline Orchestration

- [Pipeline Worker](https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py): `server/bot.py`
- [Pipeline Runner](https://github.com/sankar-s-r/voice_agent/blob/main/server/bot.py): `server/bot.py`

### Fallback Engagement

- [Fallback Observer](https://github.com/sankar-s-r/voice_agent/blob/main/server/fallback.py): `server/fallback.py`

### External Providers

- **Deepgram**
- **Groq**
- **Cartesia**

### Other components

- **Voice User**

## Connections

- Voice User → WebRTC Transport: sends audio (evidence: `README.md`)
- WebRTC Transport → Streaming STT: streams input (evidence: `server/bot.py`)
- Streaming STT → Turn Detection: transcribes (evidence: `server/bot.py`)
- Turn Detection → Streaming LLM: forwards turn (evidence: `server/bot.py`)
- Conversation Context → Streaming LLM: provides context (evidence: `server/bot.py`)
- Streaming LLM → Streaming TTS: streams response (evidence: `README.md`)
- Streaming TTS → WebRTC Transport: sends audio (evidence: `server/bot.py`)
- WebRTC Transport → Audio Output: plays audio (evidence: `README.md`)
- Pipeline Worker → WebRTC Transport: runs pipeline (evidence: `server/bot.py`)
- Pipeline Runner → Pipeline Worker: runs worker (evidence: `server/bot.py`)
- Pipeline Worker → Fallback Observer: notifies observer (evidence: `server/bot.py`)
- Fallback Observer → Pipeline Worker: queues filler (evidence: `server/fallback.py`)
- Streaming STT → Deepgram: transcribes with (evidence: `server/bot.py`)
- Streaming LLM → Groq: generates with (evidence: `server/bot.py`)
- Streaming TTS → Cartesia: synthesizes with (evidence: `server/bot.py`)
