# khalilsaqib/voxweave architecture

> Architecture diagram of the GitHub repository khalilsaqib/voxweave, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/khalilsaqib/voxweave
- Repository: https://github.com/khalilsaqib/voxweave

## Overview

VoxWeave is a web-based assistant with text chat and real-time voice conversation. Text requests stream LLM responses through the API; voice sessions join LiveKit, process microphone audio for speech recognition, generate an LLM response, segment it, synthesize speech, and publish audio. The graph includes session orchestration and conversation state as internal layers. The README documents GPU-hosted inference and streaming capabilities; some service internals were not excerpted, so those links rely on documented behavior or are omitted where wiring is uncertain.

Main entry points: [`app/services/audio_processor/service.py`](https://github.com/khalilsaqib/voxweave/blob/main/app/services/audio_processor/service.py) (Audio Processor), [`app/services/session/manager_service.py`](https://github.com/khalilsaqib/voxweave/blob/main/app/services/session/manager_service.py) (Voice Session Manager), [`app/api/routes.py`](https://github.com/khalilsaqib/voxweave/blob/main/app/api/routes.py) (HTTP API), [`app/main.py`](https://github.com/khalilsaqib/voxweave/blob/main/app/main.py) (Web Application)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Application and UI"]
  node_web_ui["Web Interface<br/>[app.js]"]
  node_app["Web Application<br/>[main.py]"]
  node_api["HTTP API<br/>[routes.py]"]
end

subgraph group_sessions["Conversation Sessions"]
  node_conversation["Conversation Manager"]
  node_session_manager["Voice Session Manager<br/>[manager_service.py]"]
  node_voice_session["Voice Session State<br/>[voice_service.py]"]
end

subgraph group_voice["Voice Processing"]
  node_livekit_service["LiveKit Service<br/>[service.py]"]
  node_audio_processor["Audio Processor<br/>[service.py]"]
  node_vad["Voice Activity Detection<br/>[service.py]"]
  node_audio_buffer["Audio Buffer<br/>[buffer.py]"]
  node_asr_scheduler["ASR Scheduler<br/>[scheduler.py]"]
  node_tts_worker["TTS Worker<br/>[worker.py]"]
  node_tts_publisher["Audio Publisher<br/>[publisher.py]"]
end

subgraph group_ai["AI Services"]
  node_llm["LLM Service<br/>[service.py]"]
  node_asr["Speech Recognition<br/>[service.py]"]
  node_segmenter["Text Segmenter<br/>[text_segmenter.py]"]
  node_tts["Speech Synthesis<br/>[service.py]"]
end

node_user(("User"))
node_llm_host{{"Hosted LLM API"}}
node_livekit{{"LiveKit WebRTC"}}
node_stt_host{{"Hosted STT API"}}
node_tts_host{{"Hosted TTS API"}}

node_user -->|"interacts with"| node_web_ui
node_web_ui -->|"uses"| node_app
node_app -->|"registers routes"| node_api
node_api -->|"manages conversations"| node_conversation
node_api -->|"starts and stops sessions"| node_session_manager
node_api -->|"creates client token"| node_livekit_service
node_api -->|"streams chat response"| node_llm
node_conversation -->|"maintains session history"| node_llm
node_app -->|"wires components"| node_session_manager
node_app -->|"wires components"| node_conversation
node_session_manager -->|"connects agent"| node_livekit_service
node_session_manager -->|"creates session state"| node_voice_session
node_session_manager -->|"dispatches audio track"| node_audio_processor
node_livekit_service -->|"joins room"| node_livekit
node_user -->|"streams microphone audio"| node_livekit
node_livekit -->|"provides audio track"| node_audio_processor
node_audio_processor -->|"detects speech"| node_vad
node_audio_processor -->|"buffers audio"| node_audio_buffer
node_audio_processor -->|"submits transcription"| node_asr_scheduler
node_asr_scheduler -->|"dispatches requests"| node_asr
node_asr -->|"transcribes audio"| node_stt_host
node_audio_processor -->|"requests voice response"| node_llm
node_llm -.->|"streams inference"| node_llm_host
node_audio_processor -->|"segments response text"| node_segmenter
node_segmenter -->|"provides text segments"| node_tts
node_tts -.->|"synthesizes speech"| node_tts_host
node_session_manager -->|"starts worker"| node_tts_worker
node_tts_worker -->|"generates segment audio"| node_tts
node_tts_worker -->|"publishes generated audio"| node_tts_publisher
node_tts_publisher -.->|"streams audio to room"| node_livekit
node_livekit -->|"plays response audio"| node_user

click node_web_ui "https://github.com/khalilsaqib/voxweave/blob/main/client/js/app.js"
click node_app "https://github.com/khalilsaqib/voxweave/blob/main/app/main.py"
click node_api "https://github.com/khalilsaqib/voxweave/blob/main/app/api/routes.py"
click node_conversation "https://github.com/khalilsaqib/voxweave/blob/main/app/services/session/conversation_service.py"
click node_session_manager "https://github.com/khalilsaqib/voxweave/blob/main/app/services/session/manager_service.py"
click node_voice_session "https://github.com/khalilsaqib/voxweave/blob/main/app/services/session/voice_service.py"
click node_llm "https://github.com/khalilsaqib/voxweave/blob/main/app/services/llm/service.py"
click node_livekit_service "https://github.com/khalilsaqib/voxweave/blob/main/app/services/livekit/service.py"
click node_audio_processor "https://github.com/khalilsaqib/voxweave/blob/main/app/services/audio_processor/service.py"
click node_vad "https://github.com/khalilsaqib/voxweave/blob/main/app/services/vad/service.py"
click node_audio_buffer "https://github.com/khalilsaqib/voxweave/blob/main/app/audio/buffer.py"
click node_asr_scheduler "https://github.com/khalilsaqib/voxweave/blob/main/app/services/asr/scheduler.py"
click node_asr "https://github.com/khalilsaqib/voxweave/blob/main/app/services/asr/service.py"
click node_segmenter "https://github.com/khalilsaqib/voxweave/blob/main/app/services/tts/text_segmenter.py"
click node_tts "https://github.com/khalilsaqib/voxweave/blob/main/app/services/tts/service.py"
click node_tts_worker "https://github.com/khalilsaqib/voxweave/blob/main/app/services/tts/worker.py"
click node_tts_publisher "https://github.com/khalilsaqib/voxweave/blob/main/app/services/tts/publisher.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web_ui,node_app,node_api,node_user toneBlue
class node_conversation,node_session_manager,node_voice_session toneAmber
class node_livekit_service,node_audio_processor,node_vad,node_audio_buffer,node_asr_scheduler,node_tts_worker,node_tts_publisher,node_llm_host,node_stt_host,node_tts_host toneMint
class node_llm,node_asr,node_segmenter,node_tts toneRose
class node_livekit toneIndigo
```

## Components

### Application and UI

- [Web Interface](https://github.com/khalilsaqib/voxweave/blob/main/client/js/app.js): `client/js/app.js`
- [Web Application](https://github.com/khalilsaqib/voxweave/blob/main/app/main.py): `app/main.py`
- [HTTP API](https://github.com/khalilsaqib/voxweave/blob/main/app/api/routes.py): `app/api/routes.py`

### Conversation Sessions

- [Conversation Manager](https://github.com/khalilsaqib/voxweave/blob/main/app/services/session/conversation_service.py): `app/services/session/conversation_service.py`
- [Voice Session Manager](https://github.com/khalilsaqib/voxweave/blob/main/app/services/session/manager_service.py): `app/services/session/manager_service.py`
- [Voice Session State](https://github.com/khalilsaqib/voxweave/blob/main/app/services/session/voice_service.py): `app/services/session/voice_service.py`

### Voice Processing

- [LiveKit Service](https://github.com/khalilsaqib/voxweave/blob/main/app/services/livekit/service.py): `app/services/livekit/service.py`
- [Audio Processor](https://github.com/khalilsaqib/voxweave/blob/main/app/services/audio_processor/service.py): `app/services/audio_processor/service.py`
- [Voice Activity Detection](https://github.com/khalilsaqib/voxweave/blob/main/app/services/vad/service.py): `app/services/vad/service.py`
- [Audio Buffer](https://github.com/khalilsaqib/voxweave/blob/main/app/audio/buffer.py): `app/audio/buffer.py`
- [ASR Scheduler](https://github.com/khalilsaqib/voxweave/blob/main/app/services/asr/scheduler.py): `app/services/asr/scheduler.py`
- [TTS Worker](https://github.com/khalilsaqib/voxweave/blob/main/app/services/tts/worker.py): `app/services/tts/worker.py`
- [Audio Publisher](https://github.com/khalilsaqib/voxweave/blob/main/app/services/tts/publisher.py): `app/services/tts/publisher.py`

### AI Services

- [LLM Service](https://github.com/khalilsaqib/voxweave/blob/main/app/services/llm/service.py): `app/services/llm/service.py`
- [Speech Recognition](https://github.com/khalilsaqib/voxweave/blob/main/app/services/asr/service.py): `app/services/asr/service.py`
- [Text Segmenter](https://github.com/khalilsaqib/voxweave/blob/main/app/services/tts/text_segmenter.py): `app/services/tts/text_segmenter.py`
- [Speech Synthesis](https://github.com/khalilsaqib/voxweave/blob/main/app/services/tts/service.py): `app/services/tts/service.py`

### Other components

- **User**
- **Hosted LLM API**
- **LiveKit WebRTC**
- **Hosted STT API**
- **Hosted TTS API**

## Connections

- User → Web Interface: interacts with (evidence: `README.md`)
- Web Interface → Web Application: uses (evidence: `README.md`)
- Web Application → HTTP API: registers routes (evidence: `app/main.py`)
- HTTP API → Conversation Manager: manages conversations (evidence: `app/api/routes.py`)
- HTTP API → Voice Session Manager: starts and stops sessions (evidence: `app/api/routes.py`)
- HTTP API → LiveKit Service: creates client token (evidence: `app/api/routes.py`)
- HTTP API → LLM Service: streams chat response (evidence: `app/api/routes.py`)
- Conversation Manager → LLM Service: maintains session history (evidence: `app/services/session/conversation_service.py`)
- Web Application → Voice Session Manager: wires components (evidence: `app/main.py`)
- Web Application → Conversation Manager: wires components (evidence: `app/main.py`)
- Voice Session Manager → LiveKit Service: connects agent (evidence: `app/services/session/manager_service.py`)
- Voice Session Manager → Voice Session State: creates session state (evidence: `app/services/session/manager_service.py`)
- Voice Session Manager → Audio Processor: dispatches audio track (evidence: `app/services/session/manager_service.py`)
- LiveKit Service → LiveKit WebRTC: joins room (evidence: `app/services/livekit/service.py`)
- User → LiveKit WebRTC: streams microphone audio (evidence: `README.md`)
- LiveKit WebRTC → Audio Processor: provides audio track
- Audio Processor → Voice Activity Detection: detects speech (evidence: `app/services/audio_processor/service.py`)
- Audio Processor → Audio Buffer: buffers audio (evidence: `app/services/audio_processor/service.py`)
- Audio Processor → ASR Scheduler: submits transcription (evidence: `app/services/audio_processor/service.py`)
- ASR Scheduler → Speech Recognition: dispatches requests (evidence: `app/services/asr/scheduler.py`)
- Speech Recognition → Hosted STT API: transcribes audio (evidence: `app/services/asr/service.py`)
- Audio Processor → LLM Service: requests voice response (evidence: `app/services/audio_processor/service.py`)
- LLM Service → Hosted LLM API: streams inference (evidence: `README.md`)
- Audio Processor → Text Segmenter: segments response text (evidence: `app/services/audio_processor/service.py`)
- Text Segmenter → Speech Synthesis: provides text segments (evidence: `README.md`)
- Speech Synthesis → Hosted TTS API: synthesizes speech (evidence: `README.md`)
- Voice Session Manager → TTS Worker: starts worker (evidence: `app/services/session/manager_service.py`)
- TTS Worker → Speech Synthesis: generates segment audio (evidence: `app/services/tts/worker.py`)
- TTS Worker → Audio Publisher: publishes generated audio (evidence: `app/services/tts/worker.py`)
- Audio Publisher → LiveKit WebRTC: streams audio to room (evidence: `README.md`)
- LiveKit WebRTC → User: plays response audio (evidence: `README.md`)