# activated-intelligence/voice-chat architecture

> Architecture diagram of the GitHub repository activated-intelligence/voice-chat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/activated-intelligence/voice-chat
- Repository: https://github.com/activated-intelligence/voice-chat

## Overview

1. This repository is a **fully browser-side voice assistant** built with **Next.js App Router + React**. There is no custom backend in the repo; the runtime boundary is the browser, with all speech, inference, and playback happening locally on the client.

2. The **main orchestration point** is `src/app/page.tsx`. It likely owns the session state, coordinates microphone capture, receives transcripts, sends prompts to the LLM, and triggers spoken responses. `layout.tsx` and `globals.css` provide the app shell and styling only.

3. The core user-facing UI is split into small components under `src/components/ui/`: `conversation.tsx`, `message.tsx`, `live-waveform.tsx`, `avatar.tsx`, and `button.tsx`. These are presentation boundaries for chat history, voice activity visualization, speaker identity/role, and primary controls.

4. The **audio ingestion pipeline** is implemented via browser-native audio processing plus shipped worker assets in `public/`. `public/vad-processor.js` is an AudioWorklet-style processor for streaming microphone audio into the VAD stage, while `public/stt-worker-esm.js` runs speech recognition logic off the main thread.

5. The **speech-to-text boundary** is local and model-driven. The README states Whisper runs via `@huggingface/transformers`, with the model downloaded on first use and cached locally. This means transcription is an asynchronous browser workload, not a server API call.

6. The **voice activity detection boundary** is separate from transcription. Silero VAD detects when the user is speaking so the app can segment utterances and avoid sending silence into STT/LLM. This is important architecturally because it gates when audio is promoted into the transcription pipeline.

7. The **LLM subsystem** is wrapped in `src/hooks/use-webllm.ts`. The repo uses **WebLLM** with a Qwen 1.5B model for local generation. The hook is the abstraction boundary between conversation state and the inference engine, making it the natural replacement point if the app is switched to an external API or a local server.

8. The **text-to-speech subsystem** is wrapped in `src/hooks/use-tts.ts` and supported by `src/lib/tts.ts`. The TTS layer handles turning generated text into audio using Supertonic via `@huggingface/transformers`, with voice assets bundled in `public/voices/*.bin`.

9. `src/lib/splitter.ts` is a small but important pipeline utility: it segments LLM output into chunks suitable for streaming speech synthesis. That makes the assistant feel responsive by allowing TTS to begin before the full response is complete.

10. `src/lib/utils.ts` is supporting infrastructure shared by the UI/components, while `components.json`, Tailwind/PostCSS config, and `tsconfig.json` define the frontend toolchain. These are secondary to the runtime architecture but confirm this is a strongly client-oriented TypeScript application.

11. The **data flow** is: microphone input → VAD → STT → conversation state → WebLLM response generation → text splitting → TTS synthesis → audio playback, while the UI renders waveform/chat updates in parallel. This is the key diagrammatic backbone of the repo.

12. A major architectural characteristic is **asset-heavy local inference**. The README calls out large model downloads from Hugging Face CDN and local caching via IndexedDB after first load. The repo’s `public/voices` binaries are prebundled, but the bigger ML models are fetched lazily in-browser and reused offline afterward.

13. The project is intentionally designed with a **replaceable LLM boundary**. The README explicitly says the voice pipeline is the main value and the LLM can be swapped for Claude, GPT, Gemini, Ollama, LM Studio, or any OpenAI-compatible endpoint. That makes the assistant pipeline modular: speech and synthesis are local, while generation can be local or remote.

14. From an architecture-diagram perspective, the most important boundaries are: browser UI, audio processing workers, model runtime hooks, bundled voice assets, and external model download sources. There is no persistent server tier in this repo; persistence is limited to local browser caches for downloaded models and assets.

Main entry points: [`src/app/page.tsx`](https://github.com/activated-intelligence/voice-chat/blob/main/src/app/page.tsx) (Voice Orchestrator), [`src/hooks/use-tts.ts`](https://github.com/activated-intelligence/voice-chat/blob/main/src/hooks/use-tts.ts) (TTS Hook), [`src/app/layout.tsx`](https://github.com/activated-intelligence/voice-chat/blob/main/src/app/layout.tsx) (App Shell), [`src/components/ui/conversation.tsx`](https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/conversation.tsx) (Conversation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["Browser App"]
  node_page["Voice Orchestrator<br/>Next.js page<br/>[page.tsx]"]
  node_layout["App Shell<br/>Next layout<br/>[layout.tsx]"]
  node_styles["Global Styles<br/>CSS<br/>[globals.css]"]
  node_conversation["Conversation<br/>UI component<br/>[conversation.tsx]"]
  node_message["Message<br/>UI component<br/>[message.tsx]"]
  node_waveform["Waveform<br/>UI component<br/>[live-waveform.tsx]"]
  node_avatar["Avatar<br/>UI component<br/>[avatar.tsx]"]
  node_button["Button<br/>UI component<br/>[button.tsx]"]
  node_utils["Shared Utils<br/>helper lib<br/>[utils.ts]"]
end

subgraph group_audio["Audio Pipeline"]
  node_vad{{"VAD Processor<br/>AudioWorklet<br/>[vad-processor.js]"}}
  node_stt{{"STT Worker<br/>Web worker<br/>[stt-worker-esm.js]"}}
end

subgraph group_ai["AI Runtime"]
  node_tts["TTS Hook<br/>React hook<br/>[use-tts.ts]"]
  node_webllm["LLM Hook<br/>React hook<br/>[use-webllm.ts]"]
  node_splitter["Text Splitter<br/>pipeline util<br/>[splitter.ts]"]
  node_ttslib["TTS Core<br/>speech lib<br/>[tts.ts]"]
end

subgraph group_assets["Bundled Assets"]
  node_voices[("Voice Assets<br/>binary voices")]
end

node_page -->|"renders"| node_conversation
node_page -->|"updates"| node_waveform
node_page -->|"renders"| node_avatar
node_page -->|"controls"| node_button
node_layout -->|"hosts"| node_page
node_layout -->|"applies"| node_styles
node_conversation -->|"contains"| node_message
node_page -->|"captures"| node_vad
node_vad -->|"gates"| node_stt
node_stt -->|"emits transcript"| node_page
node_page -->|"requests reply"| node_webllm
node_webllm -->|"streams text"| node_splitter
node_splitter -->|"feeds chunks"| node_tts
node_tts -->|"delegates"| node_ttslib
node_ttslib -->|"reads assets"| node_voices
node_tts -->|"plays audio"| node_page
node_page -->|"shares helpers"| node_utils

click node_page "https://github.com/activated-intelligence/voice-chat/blob/main/src/app/page.tsx"
click node_layout "https://github.com/activated-intelligence/voice-chat/blob/main/src/app/layout.tsx"
click node_styles "https://github.com/activated-intelligence/voice-chat/blob/main/src/app/globals.css"
click node_conversation "https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/conversation.tsx"
click node_message "https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/message.tsx"
click node_waveform "https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/live-waveform.tsx"
click node_avatar "https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/avatar.tsx"
click node_button "https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/button.tsx"
click node_vad "https://github.com/activated-intelligence/voice-chat/blob/main/public/vad-processor.js"
click node_stt "https://github.com/activated-intelligence/voice-chat/blob/main/public/stt-worker-esm.js"
click node_tts "https://github.com/activated-intelligence/voice-chat/blob/main/src/hooks/use-tts.ts"
click node_webllm "https://github.com/activated-intelligence/voice-chat/blob/main/src/hooks/use-webllm.ts"
click node_splitter "https://github.com/activated-intelligence/voice-chat/blob/main/src/lib/splitter.ts"
click node_ttslib "https://github.com/activated-intelligence/voice-chat/blob/main/src/lib/tts.ts"
click node_utils "https://github.com/activated-intelligence/voice-chat/blob/main/src/lib/utils.ts"
click node_voices "https://github.com/activated-intelligence/voice-chat/tree/main/public/voices"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_page,node_layout,node_styles,node_conversation,node_message,node_waveform,node_avatar,node_button,node_utils toneBlue
class node_vad,node_stt toneAmber
class node_tts,node_webllm,node_splitter,node_ttslib toneMint
class node_voices toneRose
```

## Components

### Browser App

Next.js client shell and orchestrator

- [Voice Orchestrator](https://github.com/activated-intelligence/voice-chat/blob/main/src/app/page.tsx): Next.js page · Main browser-side session controller and conversation flow. · `src/app/page.tsx`
- [App Shell](https://github.com/activated-intelligence/voice-chat/blob/main/src/app/layout.tsx): Next layout · Root document structure and app wrapper. · `src/app/layout.tsx`
- [Global Styles](https://github.com/activated-intelligence/voice-chat/blob/main/src/app/globals.css): CSS · Application-wide styling and visual defaults. · `src/app/globals.css`
- [Conversation](https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/conversation.tsx): UI component · Conversation history and message stream container. · `src/components/ui/conversation.tsx`
- [Message](https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/message.tsx): UI component · Single chat or assistant message presentation. · `src/components/ui/message.tsx`
- [Waveform](https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/live-waveform.tsx): UI component · Live voice activity visualization. · `src/components/ui/live-waveform.tsx`
- [Avatar](https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/avatar.tsx): UI component · Speaker identity and assistant/user visual marker. · `src/components/ui/avatar.tsx`
- [Button](https://github.com/activated-intelligence/voice-chat/blob/main/src/components/ui/button.tsx): UI component · Primary interaction control used by the voice UI. · `src/components/ui/button.tsx`
- [Shared Utils](https://github.com/activated-intelligence/voice-chat/blob/main/src/lib/utils.ts): helper lib · Common frontend utility helpers. · `src/lib/utils.ts`

### Audio Pipeline

Microphone capture, VAD, transcription, and playback

- [VAD Processor](https://github.com/activated-intelligence/voice-chat/blob/main/public/vad-processor.js): AudioWorklet · Browser audio processor that streams mic frames into voice activity detection. · `public/vad-processor.js`
- [STT Worker](https://github.com/activated-intelligence/voice-chat/blob/main/public/stt-worker-esm.js): Web worker · Off-main-thread speech recognition entrypoint. · `public/stt-worker-esm.js`

### AI Runtime

Local generation and speech synthesis hooks

- [TTS Hook](https://github.com/activated-intelligence/voice-chat/blob/main/src/hooks/use-tts.ts): React hook · Speech synthesis orchestration and playback boundary. · `src/hooks/use-tts.ts`
- [LLM Hook](https://github.com/activated-intelligence/voice-chat/blob/main/src/hooks/use-webllm.ts): React hook · Local model generation boundary using WebLLM. · `src/hooks/use-webllm.ts`
- [Text Splitter](https://github.com/activated-intelligence/voice-chat/blob/main/src/lib/splitter.ts): pipeline util · Chunks generated text for streaming speech synthesis. · `src/lib/splitter.ts`
- [TTS Core](https://github.com/activated-intelligence/voice-chat/blob/main/src/lib/tts.ts): speech lib · Supertonic-based speech synthesis helper logic. · `src/lib/tts.ts`

### Bundled Assets

Worker scripts and voice model binaries

- [Voice Assets](https://github.com/activated-intelligence/voice-chat/tree/main/public/voices): binary voices · Bundled speaker voice binaries used by local synthesis. · `public/voices`

## Connections

- Voice Orchestrator → Conversation: renders · Main session state feeds the conversation view.
- Voice Orchestrator → Waveform: updates · Voice activity state drives live waveform feedback.
- Voice Orchestrator → Avatar: renders · Speaker role and state are reflected in the avatar UI.
- Voice Orchestrator → Button: controls · Primary user actions flow through the button control.
- App Shell → Voice Orchestrator: hosts · Root layout hosts the application page.
- App Shell → Global Styles: applies · Global styles are attached through the app shell.
- Conversation → Message: contains · Conversation is composed of individual message cards.
- Voice Orchestrator → VAD Processor: captures · Microphone audio is routed into VAD processing.
- VAD Processor → STT Worker: gates · Detected speech segments are promoted into transcription.
- STT Worker → Voice Orchestrator: emits transcript · Transcribed user speech updates the orchestrator state.
- Voice Orchestrator → LLM Hook: requests reply · Transcript and conversation context are sent for generation.
- LLM Hook → Text Splitter: streams text · Generated output is segmented for incremental speaking.
- Text Splitter → TTS Hook: feeds chunks · Text chunks are sent into the TTS pipeline.
- TTS Hook → TTS Core: delegates · The hook uses lower-level synthesis helpers.
- TTS Core → Voice Assets: reads assets · Voice binaries are loaded for local synthesis.
- TTS Hook → Voice Orchestrator: plays audio · Synthesized speech returns to the assistant session.
- Voice Orchestrator → Shared Utils: shares helpers · Common utilities support the UI and orchestration code.
