# harsha-kyle/sahayak-ai architecture

> Architecture diagram of the GitHub repository harsha-kyle/sahayak-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harsha-kyle/sahayak-ai
- Repository: https://github.com/harsha-kyle/sahayak-ai

## Overview

Sahayak AI is a multilingual, voice-first rural assistance platform for agricultural, government-scheme, cooperative, and legal questions. A user selects a language and asks by voice or text; the web client calls FastAPI, which retrieves policy context, generates a grounded answer, and can return citations and speech. An ESP32 device offers a parallel audio-in/audio-out path. The graph also preserves grievance drafting, status tracking, history, analytics, and hardware monitoring. Some frontend and API subsystems are represented from documented routes and the source index; their internal behavior was not sampled.

Main entry points: [`backend/app/main.py`](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/main.py) (FastAPI entry), [`backend/app/hardware/esp32_manager.py`](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/hardware/esp32_manager.py) (Audio processing), [`src/pages/Chatbot.tsx`](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/pages/Chatbot.tsx) (Voice and text chat), [`backend/app/services/rag_service.py`](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/services/rag_service.py) (RAG orchestration)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_web["Kiosk experience"]
  node_routes["App routes<br/>[App.tsx]"]
  node_session["Session state<br/>[AppContext.tsx]"]
  node_chatui["Voice and text chat<br/>[Chatbot.tsx]"]
  node_speech["Browser speech<br/>[useSpeech.ts]"]
  node_answers["Offline answers<br/>[answers.ts]"]
  node_language["Language lookup<br/>[lookup.ts]"]
end

subgraph group_civic["Citizen services"]
  node_grievance["Grievance workflow"]
  node_grievance_data["Grievance definitions<br/>[grievance.ts]"]
  node_status["Status tracking<br/>[StatusTracker.tsx]"]
  node_analytics["Data overview<br/>[DataOverview.tsx]"]
  node_history["Session history<br/>[history.ts]"]
end

subgraph group_api["Backend API"]
  node_api_entry["FastAPI entry<br/>[main.py]"]
  node_chat_api["Chat endpoint<br/>[chat.py]"]
  node_voice_api["Voice endpoint<br/>[voice.py]"]
  node_schemes_api["Scheme endpoint<br/>[schemes.py]"]
  node_grievance_api["Grievance endpoint<br/>[grievances.py]"]
  node_documents_api["Document endpoint<br/>[documents.py]"]
end

subgraph group_ai["Answer intelligence"]
  node_rag["RAG orchestration<br/>[rag_service.py]"]
  node_vector["Vector retrieval<br/>[vector_search.py]"]
  node_keyword["Keyword retrieval<br/>[keyword_search.py]"]
  node_rerank["Candidate reranking<br/>[reranker.py]"]
  node_embed["Query embeddings<br/>[embedder.py]"]
  node_llm["Grounded generation<br/>[llm_service.py]"]
  node_citations["Citation building"]
  node_store[("Policy data store<br/>[database.py]")]
  node_stt["Speech recognition<br/>[stt_service.py]"]
  node_tts["Speech synthesis<br/>[tts_service.py]"]
end

subgraph group_device["Voice hardware"]
  node_esp32["ESP32 device"]
  node_ws["WebSocket manager"]
  node_protocol["Device protocol<br/>[protocol.py]"]
  node_device_flow["Audio processing<br/>[esp32_manager.py]"]
  node_status_panel["Hardware status"]
end

node_citizen(("Farmer / citizen"))
node_gemini["Google Gemini"]

node_citizen -->|"asks by voice/text"| node_chatui
node_routes -->|"routes to"| node_chatui
node_chatui -->|"reads session"| node_session
node_chatui -->|"uses answers"| node_answers
node_chatui -->|"uses speech"| node_speech
node_speech -->|"looks up voice"| node_language
node_chatui -->|"sends chat"| node_chat_api
node_chatui -->|"sends audio"| node_voice_api
node_routes -->|"routes to"| node_grievance
node_grievance -->|"uses definitions"| node_grievance_data
node_routes -->|"routes to"| node_status
node_routes -->|"routes to"| node_analytics
node_session -->|"loads history"| node_history
node_api_entry -->|"registers route"| node_chat_api
node_api_entry -->|"registers route"| node_voice_api
node_api_entry -->|"registers route"| node_schemes_api
node_api_entry -->|"registers route"| node_grievance_api
node_api_entry -->|"registers route"| node_documents_api
node_api_entry -->|"handles sockets"| node_ws
node_chat_api -->|"retrieves context"| node_rag
node_chat_api -->|"requests answer"| node_llm
node_rag -->|"embeds query"| node_embed
node_rag -->|"searches vectors"| node_vector
node_rag -->|"searches keywords"| node_keyword
node_rag -->|"reranks candidates"| node_rerank
node_vector -->|"reads vectors"| node_store
node_llm -.->|"generates with"| node_gemini
node_api_entry -->|"dispatches audio"| node_device_flow
node_device_flow -->|"transcribes audio"| node_stt
node_device_flow -->|"retrieves context"| node_rag
node_device_flow -->|"generates answer"| node_llm
node_device_flow -->|"builds citations"| node_citations
node_device_flow -->|"synthesizes speech"| node_tts
node_device_flow -->|"streams response"| node_ws
node_ws -->|"uses states"| node_protocol
node_esp32 -->|"sends audio"| node_api_entry
node_ws -.->|"broadcasts status"| node_status_panel
node_stt -->|"detects language"| node_language
node_tts -.->|"returns audio"| node_esp32

click node_routes "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/App.tsx"
click node_session "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/contexts/AppContext.tsx"
click node_chatui "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/pages/Chatbot.tsx"
click node_speech "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/hooks/useSpeech.ts"
click node_answers "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/data/answers.ts"
click node_language "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/utils/lookup.ts"
click node_grievance "https://github.com/harsha-kyle/sahayak-ai/tree/main/src/pages/grievance"
click node_grievance_data "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/data/grievance.ts"
click node_status "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/pages/StatusTracker.tsx"
click node_analytics "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/pages/DataOverview.tsx"
click node_history "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/data/history.ts"
click node_api_entry "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/main.py"
click node_chat_api "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/chat.py"
click node_voice_api "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/voice.py"
click node_schemes_api "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/schemes.py"
click node_grievance_api "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/grievances.py"
click node_documents_api "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/documents.py"
click node_rag "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/services/rag_service.py"
click node_vector "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/retrieval/vector_search.py"
click node_keyword "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/retrieval/keyword_search.py"
click node_rerank "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/retrieval/reranker.py"
click node_embed "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/embeddings/embedder.py"
click node_llm "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/services/llm_service.py"
click node_citations "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/citations/citation_service.py"
click node_store "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/db/database.py"
click node_stt "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/services/stt_service.py"
click node_tts "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/services/tts_service.py"
click node_ws "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/hardware/websocket_manager.py"
click node_protocol "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/hardware/protocol.py"
click node_device_flow "https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/hardware/esp32_manager.py"
click node_status_panel "https://github.com/harsha-kyle/sahayak-ai/blob/main/src/components/status/HardwareStatusPanel.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_routes,node_session,node_chatui,node_speech,node_answers,node_language toneBlue
class node_grievance,node_grievance_data,node_status,node_analytics,node_history toneAmber
class node_api_entry,node_chat_api,node_voice_api,node_schemes_api,node_grievance_api,node_documents_api toneMint
class node_rag,node_vector,node_keyword,node_rerank,node_embed,node_llm,node_citations,node_store,node_stt,node_tts toneRose
class node_esp32,node_ws,node_protocol,node_device_flow,node_status_panel,node_citizen,node_gemini toneIndigo
```

## Components

### Kiosk experience

- [App routes](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/App.tsx): `src/App.tsx`
- [Session state](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/contexts/AppContext.tsx): `src/contexts/AppContext.tsx`
- [Voice and text chat](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/pages/Chatbot.tsx): `src/pages/Chatbot.tsx`
- [Browser speech](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/hooks/useSpeech.ts): `src/hooks/useSpeech.ts`
- [Offline answers](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/data/answers.ts): `src/data/answers.ts`
- [Language lookup](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/utils/lookup.ts): `src/utils/lookup.ts`

### Citizen services

- [Grievance workflow](https://github.com/harsha-kyle/sahayak-ai/tree/main/src/pages/grievance): `src/pages/grievance`
- [Grievance definitions](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/data/grievance.ts): `src/data/grievance.ts`
- [Status tracking](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/pages/StatusTracker.tsx): `src/pages/StatusTracker.tsx`
- [Data overview](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/pages/DataOverview.tsx): `src/pages/DataOverview.tsx`
- [Session history](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/data/history.ts): `src/data/history.ts`

### Backend API

- [FastAPI entry](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/main.py): `backend/app/main.py`
- [Chat endpoint](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/chat.py): `backend/app/api/chat.py`
- [Voice endpoint](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/voice.py): `backend/app/api/voice.py`
- [Scheme endpoint](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/schemes.py): `backend/app/api/schemes.py`
- [Grievance endpoint](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/grievances.py): `backend/app/api/grievances.py`
- [Document endpoint](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/api/documents.py): `backend/app/api/documents.py`

### Answer intelligence

- [RAG orchestration](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/services/rag_service.py): `backend/app/services/rag_service.py`
- [Vector retrieval](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/retrieval/vector_search.py): `backend/app/rag/retrieval/vector_search.py`
- [Keyword retrieval](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/retrieval/keyword_search.py): `backend/app/rag/retrieval/keyword_search.py`
- [Candidate reranking](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/retrieval/reranker.py): `backend/app/rag/retrieval/reranker.py`
- [Query embeddings](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/embeddings/embedder.py): `backend/app/rag/embeddings/embedder.py`
- [Grounded generation](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/services/llm_service.py): `backend/app/services/llm_service.py`
- [Citation building](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/rag/citations/citation_service.py): `backend/app/rag/citations/citation_service.py`
- [Policy data store](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/db/database.py): `backend/app/db/database.py`
- [Speech recognition](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/services/stt_service.py): `backend/app/services/stt_service.py`
- [Speech synthesis](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/services/tts_service.py): `backend/app/services/tts_service.py`

### Voice hardware

- **ESP32 device**
- [WebSocket manager](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/hardware/websocket_manager.py): `backend/app/hardware/websocket_manager.py`
- [Device protocol](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/hardware/protocol.py): `backend/app/hardware/protocol.py`
- [Audio processing](https://github.com/harsha-kyle/sahayak-ai/blob/main/backend/app/hardware/esp32_manager.py): `backend/app/hardware/esp32_manager.py`
- [Hardware status](https://github.com/harsha-kyle/sahayak-ai/blob/main/src/components/status/HardwareStatusPanel.tsx): `src/components/status/HardwareStatusPanel.tsx`

### Other components

- **Farmer / citizen**
- **Google Gemini**

## Connections

- Farmer / citizen → Voice and text chat: asks by voice/text (evidence: `README.md`)
- App routes → Voice and text chat: routes to (evidence: `src/App.tsx`)
- Voice and text chat → Session state: reads session (evidence: `src/pages/Chatbot.tsx`)
- Voice and text chat → Offline answers: uses answers (evidence: `src/pages/Chatbot.tsx`)
- Voice and text chat → Browser speech: uses speech (evidence: `src/pages/Chatbot.tsx`)
- Browser speech → Language lookup: looks up voice (evidence: `src/hooks/useSpeech.ts`)
- Voice and text chat → Chat endpoint: sends chat (evidence: `src/pages/Chatbot.tsx`)
- Voice and text chat → Voice endpoint: sends audio (evidence: `src/pages/Chatbot.tsx`)
- App routes → Grievance workflow: routes to (evidence: `src/App.tsx`)
- Grievance workflow → Grievance definitions: uses definitions (evidence: `src/pages/grievance/GrievanceDraft.tsx`)
- App routes → Status tracking: routes to (evidence: `src/App.tsx`)
- App routes → Data overview: routes to (evidence: `src/App.tsx`)
- Session state → Session history: loads history (evidence: `src/contexts/AppContext.tsx`)
- FastAPI entry → Chat endpoint: registers route (evidence: `backend/app/main.py`)
- FastAPI entry → Voice endpoint: registers route (evidence: `backend/app/main.py`)
- FastAPI entry → Scheme endpoint: registers route (evidence: `backend/app/main.py`)
- FastAPI entry → Grievance endpoint: registers route (evidence: `backend/app/main.py`)
- FastAPI entry → Document endpoint: registers route (evidence: `backend/app/main.py`)
- FastAPI entry → WebSocket manager: handles sockets (evidence: `backend/app/main.py`)
- Chat endpoint → RAG orchestration: retrieves context (evidence: `backend/app/api/chat.py`)
- Chat endpoint → Grounded generation: requests answer (evidence: `backend/app/api/chat.py`)
- RAG orchestration → Query embeddings: embeds query (evidence: `backend/app/services/rag_service.py`)
- RAG orchestration → Vector retrieval: searches vectors (evidence: `backend/app/services/rag_service.py`)
- RAG orchestration → Keyword retrieval: searches keywords (evidence: `backend/app/services/rag_service.py`)
- RAG orchestration → Candidate reranking: reranks candidates (evidence: `backend/app/services/rag_service.py`)
- Vector retrieval → Policy data store: reads vectors (evidence: `backend/app/rag/retrieval/vector_search.py`)
- Grounded generation → Google Gemini: generates with (evidence: `README.md`)
- FastAPI entry → Audio processing: dispatches audio (evidence: `backend/app/main.py`)
- Audio processing → Speech recognition: transcribes audio (evidence: `backend/app/hardware/esp32_manager.py`)
- Audio processing → RAG orchestration: retrieves context (evidence: `backend/app/hardware/esp32_manager.py`)
- Audio processing → Grounded generation: generates answer (evidence: `backend/app/hardware/esp32_manager.py`)
- Audio processing → Citation building: builds citations (evidence: `backend/app/hardware/esp32_manager.py`)
- Audio processing → Speech synthesis: synthesizes speech (evidence: `backend/app/hardware/esp32_manager.py`)
- Audio processing → WebSocket manager: streams response (evidence: `backend/app/hardware/esp32_manager.py`)
- WebSocket manager → Device protocol: uses states (evidence: `backend/app/hardware/websocket_manager.py`)
- ESP32 device → FastAPI entry: sends audio (evidence: `backend/app/main.py`)
- WebSocket manager → Hardware status: broadcasts status (evidence: `README.md`)
- Speech recognition → Language lookup: detects language (evidence: `backend/app/services/stt_service.py`)
- Speech synthesis → ESP32 device: returns audio (evidence: `README.md`)
