# salesforceairesearch/voiceagentrag architecture

> Architecture diagram of the GitHub repository salesforceairesearch/voiceagentrag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/salesforceairesearch/voiceagentrag
- Repository: https://github.com/salesforceairesearch/voiceagentrag

## Overview

VoiceAgentRAG is a real-time voice-oriented RAG library whose central workflow routes a query through a foreground Fast Talker while a background Slow Thinker prefetches context into a semantic cache. The router also supports document ingestion, streamed responses, and metrics; the README documents optional speech input/output and multiple model and vector-store providers. Relationships below follow the README and shown excerpts or SOURCE INDEX references; voice wiring and some optional provider integrations are not shown in source excerpts.

Main entry points: [`voice_optimized_rag/core/memory_router.py`](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/memory_router.py) (Memory Router), [`voice_optimized_rag/core/fast_talker.py`](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/fast_talker.py) (Fast Talker), [`voice_optimized_rag/core/slow_thinker.py`](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/slow_thinker.py) (Slow Thinker), [`voice_optimized_rag/llm/base.py`](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/base.py) (LLM Provider Factory)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Conversation orchestration"]
  node_router["Memory Router<br/>[memory_router.py]"]
  node_stream["Conversation Stream"]
  node_slow["Slow Thinker<br/>[slow_thinker.py]"]
  node_fast["Fast Talker<br/>[fast_talker.py]"]
end

subgraph group_retrieval["Retrieval and memory"]
  node_cache[("Semantic Cache<br/>[semantic_cache.py]")]
  node_loader["Document Loader<br/>[document_loader.py]"]
  node_embeddings["Embedding Providers<br/>[embeddings.py]"]
  node_faiss[("FAISS Store<br/>[vector_store.py]")]
  node_qdrant[("Qdrant Store<br/>[qdrant_store.py]")]
end

subgraph group_models["Model providers"]
  node_llm_factory["LLM Provider Factory<br/>[base.py]"]
  node_openai["OpenAI Provider<br/>[openai_provider.py]"]
  node_anthropic["Anthropic Provider"]
  node_gemini["Gemini Provider<br/>[gemini_provider.py]"]
  node_ollama["Ollama Provider<br/>[ollama_provider.py]"]
end

subgraph group_voice["Voice interface"]
  node_audio["Audio Stream<br/>[audio_stream.py]"]
  node_stt["Speech Recognition<br/>[stt.py]"]
  node_tts["Speech Synthesis<br/>[tts.py]"]
end

subgraph group_operations["Configuration and metrics"]
  node_config["VOR Config<br/>[config.py]"]
  node_metrics["Metrics Collector<br/>[metrics.py]"]
end

node_user(("User"))
node_llm_service{{"Model Services"}}

node_user -->|"asks"| node_router
node_router -->|"publishes events"| node_stream
node_stream -->|"delivers events"| node_slow
node_router -->|"starts"| node_slow
node_router -->|"routes queries"| node_fast
node_slow -->|"predicts topics"| node_llm_factory
node_slow -->|"embeds queries"| node_embeddings
node_slow -->|"retrieves context"| node_faiss
node_slow -->|"prefills"| node_cache
node_fast -->|"embeds queries"| node_embeddings
node_fast -->|"checks context"| node_cache
node_fast -->|"retrieves on miss"| node_faiss
node_fast -->|"signals priority"| node_stream
node_fast -->|"generates response"| node_llm_factory
node_router -->|"loads documents"| node_loader
node_router -->|"embeds documents"| node_embeddings
node_router -->|"stores documents"| node_faiss
node_router -.->|"selects store"| node_qdrant
node_llm_factory -->|"creates"| node_openai
node_llm_factory -->|"creates"| node_anthropic
node_llm_factory -->|"creates"| node_gemini
node_llm_factory -->|"creates"| node_ollama
node_openai -.->|"calls model API"| node_llm_service
node_anthropic -.->|"calls model API"| node_llm_service
node_gemini -.->|"calls model API"| node_llm_service
node_ollama -.->|"calls model API"| node_llm_service
node_router -->|"exposes metrics"| node_metrics
node_router -->|"uses settings"| node_config
node_user -.->|"provides speech"| node_audio
node_audio -.->|"streams audio"| node_stt
node_stt -.->|"transcribes query"| node_router
node_router -.->|"returns speech"| node_tts
node_tts -.->|"speaks response"| node_user

click node_router "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/memory_router.py"
click node_stream "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/conversation_stream.py"
click node_slow "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/slow_thinker.py"
click node_fast "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/fast_talker.py"
click node_cache "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/semantic_cache.py"
click node_loader "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/retrieval/document_loader.py"
click node_embeddings "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/retrieval/embeddings.py"
click node_faiss "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/retrieval/vector_store.py"
click node_qdrant "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/retrieval/qdrant_store.py"
click node_llm_factory "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/base.py"
click node_openai "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/openai_provider.py"
click node_anthropic "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/anthropic_provider.py"
click node_gemini "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/gemini_provider.py"
click node_ollama "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/ollama_provider.py"
click node_audio "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/voice/audio_stream.py"
click node_stt "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/voice/stt.py"
click node_tts "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/voice/tts.py"
click node_config "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/config.py"
click node_metrics "https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/utils/metrics.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_router,node_stream,node_slow,node_fast,node_user toneBlue
class node_cache,node_loader,node_embeddings,node_faiss,node_qdrant toneAmber
class node_llm_factory,node_openai,node_anthropic,node_gemini,node_ollama toneMint
class node_audio,node_stt,node_tts toneRose
class node_config,node_metrics,node_llm_service toneIndigo
```

## Components

### Conversation orchestration

- [Memory Router](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/memory_router.py): `voice_optimized_rag/core/memory_router.py`
- [Conversation Stream](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/conversation_stream.py): `voice_optimized_rag/core/conversation_stream.py`
- [Slow Thinker](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/slow_thinker.py): `voice_optimized_rag/core/slow_thinker.py`
- [Fast Talker](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/fast_talker.py): `voice_optimized_rag/core/fast_talker.py`

### Retrieval and memory

- [Semantic Cache](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/core/semantic_cache.py): `voice_optimized_rag/core/semantic_cache.py`
- [Document Loader](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/retrieval/document_loader.py): `voice_optimized_rag/retrieval/document_loader.py`
- [Embedding Providers](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/retrieval/embeddings.py): `voice_optimized_rag/retrieval/embeddings.py`
- [FAISS Store](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/retrieval/vector_store.py): `voice_optimized_rag/retrieval/vector_store.py`
- [Qdrant Store](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/retrieval/qdrant_store.py): `voice_optimized_rag/retrieval/qdrant_store.py`

### Model providers

- [LLM Provider Factory](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/base.py): `voice_optimized_rag/llm/base.py`
- [OpenAI Provider](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/openai_provider.py): `voice_optimized_rag/llm/openai_provider.py`
- [Anthropic Provider](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/anthropic_provider.py): `voice_optimized_rag/llm/anthropic_provider.py`
- [Gemini Provider](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/gemini_provider.py): `voice_optimized_rag/llm/gemini_provider.py`
- [Ollama Provider](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/llm/ollama_provider.py): `voice_optimized_rag/llm/ollama_provider.py`

### Voice interface

- [Audio Stream](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/voice/audio_stream.py): `voice_optimized_rag/voice/audio_stream.py`
- [Speech Recognition](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/voice/stt.py): `voice_optimized_rag/voice/stt.py`
- [Speech Synthesis](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/voice/tts.py): `voice_optimized_rag/voice/tts.py`

### Configuration and metrics

- [VOR Config](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/config.py): `voice_optimized_rag/config.py`
- [Metrics Collector](https://github.com/salesforceairesearch/voiceagentrag/blob/main/voice_optimized_rag/utils/metrics.py): `voice_optimized_rag/utils/metrics.py`

### Other components

- **User**
- **Model Services**

## Connections

- User → Memory Router: asks (evidence: `README.md`)
- Memory Router → Conversation Stream: publishes events (evidence: `voice_optimized_rag/core/memory_router.py`)
- Conversation Stream → Slow Thinker: delivers events (evidence: `README.md`)
- Memory Router → Slow Thinker: starts (evidence: `voice_optimized_rag/core/memory_router.py`)
- Memory Router → Fast Talker: routes queries (evidence: `voice_optimized_rag/core/memory_router.py`)
- Slow Thinker → LLM Provider Factory: predicts topics (evidence: `voice_optimized_rag/core/slow_thinker.py`)
- Slow Thinker → Embedding Providers: embeds queries (evidence: `voice_optimized_rag/core/slow_thinker.py`)
- Slow Thinker → FAISS Store: retrieves context (evidence: `voice_optimized_rag/core/slow_thinker.py`)
- Slow Thinker → Semantic Cache: prefills (evidence: `voice_optimized_rag/core/slow_thinker.py`)
- Fast Talker → Embedding Providers: embeds queries (evidence: `voice_optimized_rag/core/fast_talker.py`)
- Fast Talker → Semantic Cache: checks context (evidence: `voice_optimized_rag/core/fast_talker.py`)
- Fast Talker → FAISS Store: retrieves on miss (evidence: `voice_optimized_rag/core/fast_talker.py`)
- Fast Talker → Conversation Stream: signals priority (evidence: `voice_optimized_rag/core/fast_talker.py`)
- Fast Talker → LLM Provider Factory: generates response (evidence: `voice_optimized_rag/core/fast_talker.py`)
- Memory Router → Document Loader: loads documents (evidence: `voice_optimized_rag/core/memory_router.py`)
- Memory Router → Embedding Providers: embeds documents (evidence: `voice_optimized_rag/core/memory_router.py`)
- Memory Router → FAISS Store: stores documents (evidence: `voice_optimized_rag/core/memory_router.py`)
- Memory Router → Qdrant Store: selects store (evidence: `voice_optimized_rag/core/memory_router.py`)
- LLM Provider Factory → OpenAI Provider: creates (evidence: `voice_optimized_rag/llm/base.py`)
- LLM Provider Factory → Anthropic Provider: creates (evidence: `voice_optimized_rag/llm/base.py`)
- LLM Provider Factory → Gemini Provider: creates (evidence: `voice_optimized_rag/llm/base.py`)
- LLM Provider Factory → Ollama Provider: creates (evidence: `voice_optimized_rag/llm/base.py`)
- OpenAI Provider → Model Services: calls model API
- Anthropic Provider → Model Services: calls model API
- Gemini Provider → Model Services: calls model API
- Ollama Provider → Model Services: calls model API
- Memory Router → Metrics Collector: exposes metrics (evidence: `README.md`)
- Memory Router → VOR Config: uses settings (evidence: `voice_optimized_rag/core/memory_router.py`)
- User → Audio Stream: provides speech
- Audio Stream → Speech Recognition: streams audio
- Speech Recognition → Memory Router: transcribes query
- Memory Router → Speech Synthesis: returns speech
- Speech Synthesis → User: speaks response
