How tjmhmdaarif/voice-rag-pipeline works
Voice RAG accepts audio or text, transcribes audio when supplied, retrieves relevant passages using a selectable chunking strategy, synthesizes an answer from retrieved context, and applies grounding and safety checks before returning a response. The documented product also exposes chunk-strategy comparison and latency benchmarking, alongside a browser interface with audio-reactive visuals. The harness and API wiring are supported by sampled source; benchmark internals and some frontend request handling were not fully sampled, so uncertain links are omitted.
Main entry points: agent_harness.py Pipeline harness · main.py FastAPI endpoints · index.py In-memory vector index · stt_service.py STT dispatcher
Components 16
Voice interface
- Voice RAG UI
frontend/index.html - Audio and UI logic
frontend/js/app.js
API and orchestration
- FastAPI endpoints
backend/main.py - Pipeline harness
backend/harness/agent_harness.py - Answer synthesis
backend/harness/agent_harness.py
Retrieval pipeline
- STT dispatcher
backend/stt/stt_service.py - Passage loader
backend/dataset_loader.py - Chunking strategies
backend/chunking/strategies.py - Chunk model
backend/chunking/base.py - In-memory vector index
backend/vector_db/index.py
Validation and analytics
- Safety and grounding
backend/guardrails/guardrail_manager.py - Latency benchmark
backend/analytics/benchmark.py - Latency statistics
backend/analytics/latency_tracker.py
Other components
- User
- Sarvam STT
- ElevenLabs STT
Connections 15
- User → Voice RAG UI: uses
- Voice RAG UI → Audio and UI logic: runs
- Audio and UI logic → FastAPI endpoints: submits query
- FastAPI endpoints → Pipeline harness: invokes pipeline
- Pipeline harness → STT dispatcher: transcribes audio
- STT dispatcher → Sarvam STT: calls provider
- STT dispatcher → ElevenLabs STT: calls provider
- Pipeline harness → In-memory vector index: searches passages
- In-memory vector index → Passage loader: loads passages
- In-memory vector index → Chunking strategies: builds chunks
- Chunking strategies → Chunk model: creates chunks
- Pipeline harness → Answer synthesis: synthesizes answer
- Pipeline harness → Safety and grounding: validates answer
- FastAPI endpoints → Latency benchmark: starts benchmark
- FastAPI endpoints → In-memory vector index: compares strategies