# tjmhmdaarif/voice-rag-pipeline architecture

> Architecture diagram of the GitHub repository tjmhmdaarif/voice-rag-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tjmhmdaarif/voice-rag-pipeline
- Repository: https://github.com/tjmhmdaarif/voice-rag-pipeline

## Overview

Voice RAG accepts audio or text, transcribes audio when supplied, retrieves relevant passages using a selectable chunking strategy, synthesizes an answer from retrieved context, and applies grounding and safety checks before returning a response. The documented product also exposes chunk-strategy comparison and latency benchmarking, alongside a browser interface with audio-reactive visuals. The harness and API wiring are supported by sampled source; benchmark internals and some frontend request handling were not fully sampled, so uncertain links are omitted.

Main entry points: [`backend/harness/agent_harness.py`](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/harness/agent_harness.py) (Pipeline harness), [`backend/main.py`](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/main.py) (FastAPI endpoints), [`backend/vector_db/index.py`](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/vector_db/index.py) (In-memory vector index), [`backend/stt/stt_service.py`](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/stt/stt_service.py) (STT dispatcher)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experience["Voice interface"]
  node_browser["Voice RAG UI<br/>[index.html]"]
  node_frontend_logic["Audio and UI logic<br/>[app.js]"]
end

subgraph group_api["API and orchestration"]
  node_api_app["FastAPI endpoints<br/>[main.py]"]
  node_harness["Pipeline harness<br/>[agent_harness.py]"]
  node_answer["Answer synthesis<br/>[agent_harness.py]"]
end

subgraph group_retrieval["Retrieval pipeline"]
  node_stt["STT dispatcher<br/>[stt_service.py]"]
  node_dataset["Passage loader<br/>[dataset_loader.py]"]
  node_chunkers["Chunking strategies<br/>[strategies.py]"]
  node_chunk_model["Chunk model<br/>[base.py]"]
  node_vector_index[("In-memory vector index<br/>[index.py]")]
end

subgraph group_quality["Validation and analytics"]
  node_guardrails["Safety and grounding"]
  node_benchmark["Latency benchmark<br/>[benchmark.py]"]
  node_latency["Latency statistics<br/>[latency_tracker.py]"]
end

node_user(("User"))
node_sarvam["Sarvam STT"]
node_elevenlabs["ElevenLabs STT"]

node_user -->|"uses"| node_browser
node_browser -->|"runs"| node_frontend_logic
node_frontend_logic -->|"submits query"| node_api_app
node_api_app -->|"invokes pipeline"| node_harness
node_harness -->|"transcribes audio"| node_stt
node_stt -.->|"calls provider"| node_sarvam
node_stt -.->|"calls provider"| node_elevenlabs
node_harness -->|"searches passages"| node_vector_index
node_vector_index -->|"loads passages"| node_dataset
node_vector_index -->|"builds chunks"| node_chunkers
node_chunkers -->|"creates chunks"| node_chunk_model
node_harness -->|"synthesizes answer"| node_answer
node_harness -->|"validates answer"| node_guardrails
node_api_app -->|"starts benchmark"| node_benchmark
node_api_app -->|"compares strategies"| node_vector_index

click node_browser "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/frontend/index.html"
click node_frontend_logic "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/frontend/js/app.js"
click node_api_app "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/main.py"
click node_harness "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/harness/agent_harness.py"
click node_stt "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/stt/stt_service.py"
click node_dataset "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/dataset_loader.py"
click node_chunkers "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/chunking/strategies.py"
click node_chunk_model "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/chunking/base.py"
click node_vector_index "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/vector_db/index.py"
click node_answer "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/harness/agent_harness.py"
click node_guardrails "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/guardrails/guardrail_manager.py"
click node_benchmark "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/analytics/benchmark.py"
click node_latency "https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/analytics/latency_tracker.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser,node_frontend_logic,node_user toneBlue
class node_api_app,node_harness,node_answer toneAmber
class node_stt,node_dataset,node_chunkers,node_chunk_model,node_vector_index toneMint
class node_guardrails,node_benchmark,node_latency toneRose
class node_sarvam,node_elevenlabs toneIndigo
```

## Components

### Voice interface

- [Voice RAG UI](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/frontend/index.html): `frontend/index.html`
- [Audio and UI logic](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/frontend/js/app.js): `frontend/js/app.js`

### API and orchestration

- [FastAPI endpoints](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/main.py): `backend/main.py`
- [Pipeline harness](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/harness/agent_harness.py): `backend/harness/agent_harness.py`
- [Answer synthesis](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/harness/agent_harness.py): `backend/harness/agent_harness.py`

### Retrieval pipeline

- [STT dispatcher](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/stt/stt_service.py): `backend/stt/stt_service.py`
- [Passage loader](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/dataset_loader.py): `backend/dataset_loader.py`
- [Chunking strategies](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/chunking/strategies.py): `backend/chunking/strategies.py`
- [Chunk model](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/chunking/base.py): `backend/chunking/base.py`
- [In-memory vector index](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/vector_db/index.py): `backend/vector_db/index.py`

### Validation and analytics

- [Safety and grounding](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/guardrails/guardrail_manager.py): `backend/guardrails/guardrail_manager.py`
- [Latency benchmark](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/analytics/benchmark.py): `backend/analytics/benchmark.py`
- [Latency statistics](https://github.com/tjmhmdaarif/voice-rag-pipeline/blob/main/backend/analytics/latency_tracker.py): `backend/analytics/latency_tracker.py`

### Other components

- **User**
- **Sarvam STT**
- **ElevenLabs STT**

## Connections

- User → Voice RAG UI: uses
- Voice RAG UI → Audio and UI logic: runs
- Audio and UI logic → FastAPI endpoints: submits query
- FastAPI endpoints → Pipeline harness: invokes pipeline
- Pipeline harness → STT dispatcher: transcribes audio
- STT dispatcher → Sarvam STT: calls provider
- STT dispatcher → ElevenLabs STT: calls provider
- Pipeline harness → In-memory vector index: searches passages
- In-memory vector index → Passage loader: loads passages
- In-memory vector index → Chunking strategies: builds chunks
- Chunking strategies → Chunk model: creates chunks
- Pipeline harness → Answer synthesis: synthesizes answer
- Pipeline harness → Safety and grounding: validates answer
- FastAPI endpoints → Latency benchmark: starts benchmark
- FastAPI endpoints → In-memory vector index: compares strategies