# sk9315/vaani architecture

> Architecture diagram of the GitHub repository sk9315/vaani, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sk9315/vaani
- Repository: https://github.com/sk9315/vaani

## Overview

This is a LiveKit Agents Python starter for a multi-participant voice assistant, with the Vaani browser client as its local development experience. A person joins a room, speaks or sends text, and the agent routes the turn to a bot for a spoken response; the development client also supports voice selection and preview. The graph includes room context, voice configuration, and the documented model, observability, and audio-cancellation capabilities. Some subsystem links rely on README statements or source-index references; unsampled implementation details are not asserted.

Main entry points: [`src/agent.py`](https://github.com/sk9315/vaani/blob/main/src/agent.py) (LiveKit worker), [`src/dev_client/app.js`](https://github.com/sk9315/vaani/blob/main/src/dev_client/app.js) (Vaani client), [`src/voice_pipeline.py`](https://github.com/sk9315/vaani/blob/main/src/voice_pipeline.py) (Voice turn processing), [`src/dev_server.py`](https://github.com/sk9315/vaani/blob/main/src/dev_server.py) (Dev token server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Browser client"]
  node_browser["Vaani client<br/>[app.js]"]
  node_client_state["Client presentation<br/>[ui_state.js]"]
  node_orb["Voice orb<br/>[orb.js]"]
  node_token_server["Dev token server<br/>[dev_server.py]"]
end

subgraph group_runtime["Agent runtime"]
  node_worker["LiveKit worker<br/>[agent.py]"]
  node_turn_processor["Voice turn processing<br/>[voice_pipeline.py]"]
end

subgraph group_room["Room coordination"]
  node_coordinator["Turn routing"]
  node_context["Room context<br/>[room_context.py]"]
  node_room_events["Room UI payloads<br/>[room_ui.py]"]
end

subgraph group_voice["Voice services"]
  node_voice_config["Voice profiles<br/>[voice_config.py]"]
end

node_person(("Participant"))
node_livekit{{"LiveKit Cloud"}}
node_inference{{"Model inference"}}
node_stt{{"Speech recognition"}}
node_tts{{"Speech synthesis"}}
node_cancellation{{"Background cancellation"}}
node_observability{{"Agent observability"}}

node_person -->|"uses"| node_browser
node_browser -->|"renders via"| node_client_state
node_browser -.->|"displays"| node_orb
node_browser -.->|"requests token"| node_token_server
node_person -->|"requests token"| node_token_server
node_browser -->|"joins room"| node_livekit
node_livekit -->|"dispatches session"| node_worker
node_worker -->|"routes turns"| node_coordinator
node_worker -->|"processes speech"| node_turn_processor
node_turn_processor -->|"submits transcript"| node_coordinator
node_coordinator -->|"records turns"| node_context
node_worker -->|"publishes room state"| node_room_events
node_room_events -->|"reads snapshot"| node_context
node_worker -->|"configures voices"| node_voice_config
node_token_server -->|"validates and previews"| node_voice_config
node_worker -->|"runs model pipeline"| node_inference
node_turn_processor -->|"consumes speech events"| node_stt
node_worker -->|"synthesizes replies"| node_tts
node_worker -->|"cancels background voice"| node_cancellation
node_worker -->|"reports session insights"| node_observability

click node_browser "https://github.com/sk9315/vaani/blob/main/src/dev_client/app.js"
click node_client_state "https://github.com/sk9315/vaani/blob/main/src/dev_client/ui_state.js"
click node_orb "https://github.com/sk9315/vaani/blob/main/src/dev_client/orb.js"
click node_token_server "https://github.com/sk9315/vaani/blob/main/src/dev_server.py"
click node_worker "https://github.com/sk9315/vaani/blob/main/src/agent.py"
click node_turn_processor "https://github.com/sk9315/vaani/blob/main/src/voice_pipeline.py"
click node_coordinator "https://github.com/sk9315/vaani/blob/main/src/room_coordinator.py"
click node_context "https://github.com/sk9315/vaani/blob/main/src/room_context.py"
click node_room_events "https://github.com/sk9315/vaani/blob/main/src/room_ui.py"
click node_voice_config "https://github.com/sk9315/vaani/blob/main/src/voice_config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser,node_client_state,node_orb,node_token_server toneBlue
class node_worker,node_turn_processor toneAmber
class node_coordinator,node_context,node_room_events toneMint
class node_voice_config,node_cancellation toneRose
class node_person,node_livekit,node_inference,node_stt,node_tts,node_observability toneIndigo
```

## Components

### Browser client

- [Vaani client](https://github.com/sk9315/vaani/blob/main/src/dev_client/app.js): `src/dev_client/app.js`
- [Client presentation](https://github.com/sk9315/vaani/blob/main/src/dev_client/ui_state.js): `src/dev_client/ui_state.js`
- [Voice orb](https://github.com/sk9315/vaani/blob/main/src/dev_client/orb.js): `src/dev_client/orb.js`
- [Dev token server](https://github.com/sk9315/vaani/blob/main/src/dev_server.py): `src/dev_server.py`

### Agent runtime

- [LiveKit worker](https://github.com/sk9315/vaani/blob/main/src/agent.py): `src/agent.py`
- [Voice turn processing](https://github.com/sk9315/vaani/blob/main/src/voice_pipeline.py): `src/voice_pipeline.py`

### Room coordination

- [Turn routing](https://github.com/sk9315/vaani/blob/main/src/room_coordinator.py): `src/room_coordinator.py`
- [Room context](https://github.com/sk9315/vaani/blob/main/src/room_context.py): `src/room_context.py`
- [Room UI payloads](https://github.com/sk9315/vaani/blob/main/src/room_ui.py): `src/room_ui.py`

### Voice services

- [Voice profiles](https://github.com/sk9315/vaani/blob/main/src/voice_config.py): `src/voice_config.py`

### Other components

- **Participant**
- **LiveKit Cloud**
- **Model inference**
- **Speech recognition**
- **Speech synthesis**
- **Background cancellation**
- **Agent observability**

## Connections

- Participant → Vaani client: uses (evidence: `README.md`)
- Vaani client → Client presentation: renders via (evidence: `src/dev_client/app.js`)
- Vaani client → Voice orb: displays
- Vaani client → Dev token server: requests token
- Participant → Dev token server: requests token (evidence: `README.md`)
- Vaani client → LiveKit Cloud: joins room (evidence: `README.md`)
- LiveKit Cloud → LiveKit worker: dispatches session (evidence: `README.md`)
- LiveKit worker → Turn routing: routes turns (evidence: `src/agent.py`)
- LiveKit worker → Voice turn processing: processes speech (evidence: `src/agent.py`)
- Voice turn processing → Turn routing: submits transcript (evidence: `src/voice_pipeline.py`)
- Turn routing → Room context: records turns (evidence: `src/room_coordinator.py`)
- LiveKit worker → Room UI payloads: publishes room state (evidence: `src/agent.py`)
- Room UI payloads → Room context: reads snapshot (evidence: `src/room_ui.py`)
- LiveKit worker → Voice profiles: configures voices (evidence: `src/agent.py`)
- Dev token server → Voice profiles: validates and previews (evidence: `src/dev_server.py`)
- LiveKit worker → Model inference: runs model pipeline (evidence: `README.md`)
- Voice turn processing → Speech recognition: consumes speech events (evidence: `src/voice_pipeline.py`)
- LiveKit worker → Speech synthesis: synthesizes replies (evidence: `README.md`)
- LiveKit worker → Background cancellation: cancels background voice (evidence: `README.md`)
- LiveKit worker → Agent observability: reports session insights (evidence: `README.md`)
