How salesforceairesearch/voiceagentrag works
VoiceAgentRAG is a real-time voice-oriented RAG library whose central workflow routes a query through a foreground Fast Talker while a background Slow Thinker prefetches context into a semantic cache. The router also supports document ingestion, streamed responses, and metrics; the README documents optional speech input/output and multiple model and vector-store providers. Relationships below follow the README and shown excerpts or SOURCE INDEX references; voice wiring and some optional provider integrations are not shown in source excerpts.
Main entry points: memory_router.py Memory Router · fast_talker.py Fast Talker · slow_thinker.py Slow Thinker · base.py LLM Provider Factory
Components 21
Conversation orchestration
- Memory Router
voice_optimized_rag/core/memory_router.py - Conversation Stream
voice_optimized_rag/core/conversation_stream.py - Slow Thinker
voice_optimized_rag/core/slow_thinker.py - Fast Talker
voice_optimized_rag/core/fast_talker.py
Retrieval and memory
- Semantic Cache
voice_optimized_rag/core/semantic_cache.py - Document Loader
voice_optimized_rag/retrieval/document_loader.py - Embedding Providers
voice_optimized_rag/retrieval/embeddings.py - FAISS Store
voice_optimized_rag/retrieval/vector_store.py - Qdrant Store
voice_optimized_rag/retrieval/qdrant_store.py
Model providers
- LLM Provider Factory
voice_optimized_rag/llm/base.py - OpenAI Provider
voice_optimized_rag/llm/openai_provider.py - Anthropic Provider
voice_optimized_rag/llm/anthropic_provider.py - Gemini Provider
voice_optimized_rag/llm/gemini_provider.py - Ollama Provider
voice_optimized_rag/llm/ollama_provider.py
Voice interface
- Audio Stream
voice_optimized_rag/voice/audio_stream.py - Speech Recognition
voice_optimized_rag/voice/stt.py - Speech Synthesis
voice_optimized_rag/voice/tts.py
Configuration and metrics
- VOR Config
voice_optimized_rag/config.py - Metrics Collector
voice_optimized_rag/utils/metrics.py
Other components
- User
- Model Services
Connections 33
- User → Memory Router: asks
README.md - Memory Router → Conversation Stream: publishes events
voice_optimized_rag/core/memory_router.py - Conversation Stream → Slow Thinker: delivers events
README.md - Memory Router → Slow Thinker: starts
voice_optimized_rag/core/memory_router.py - Memory Router → Fast Talker: routes queries
voice_optimized_rag/core/memory_router.py - Slow Thinker → LLM Provider Factory: predicts topics
voice_optimized_rag/core/slow_thinker.py - Slow Thinker → Embedding Providers: embeds queries
voice_optimized_rag/core/slow_thinker.py - Slow Thinker → FAISS Store: retrieves context
voice_optimized_rag/core/slow_thinker.py - Slow Thinker → Semantic Cache: prefills
voice_optimized_rag/core/slow_thinker.py - Fast Talker → Embedding Providers: embeds queries
voice_optimized_rag/core/fast_talker.py - Fast Talker → Semantic Cache: checks context
voice_optimized_rag/core/fast_talker.py - Fast Talker → FAISS Store: retrieves on miss
voice_optimized_rag/core/fast_talker.py - Fast Talker → Conversation Stream: signals priority
voice_optimized_rag/core/fast_talker.py - Fast Talker → LLM Provider Factory: generates response
voice_optimized_rag/core/fast_talker.py - Memory Router → Document Loader: loads documents
voice_optimized_rag/core/memory_router.py - Memory Router → Embedding Providers: embeds documents
voice_optimized_rag/core/memory_router.py - Memory Router → FAISS Store: stores documents
voice_optimized_rag/core/memory_router.py - Memory Router → Qdrant Store: selects store
voice_optimized_rag/core/memory_router.py - LLM Provider Factory → OpenAI Provider: creates
voice_optimized_rag/llm/base.py - LLM Provider Factory → Anthropic Provider: creates
voice_optimized_rag/llm/base.py - LLM Provider Factory → Gemini Provider: creates
voice_optimized_rag/llm/base.py - LLM Provider Factory → Ollama Provider: creates
voice_optimized_rag/llm/base.py - OpenAI Provider → Model Services: calls model API
- Anthropic Provider → Model Services: calls model API
- Gemini Provider → Model Services: calls model API
- Ollama Provider → Model Services: calls model API
- Memory Router → Metrics Collector: exposes metrics
README.md - Memory Router → VOR Config: uses settings
voice_optimized_rag/core/memory_router.py - User → Audio Stream: provides speech
- Audio Stream → Speech Recognition: streams audio
- Speech Recognition → Memory Router: transcribes query
- Memory Router → Speech Synthesis: returns speech
- Speech Synthesis → User: speaks response