How harsha-kyle/sahayak-ai works
Sahayak AI is a multilingual, voice-first rural assistance platform for agricultural, government-scheme, cooperative, and legal questions. A user selects a language and asks by voice or text; the web client calls FastAPI, which retrieves policy context, generates a grounded answer, and can return citations and speech. An ESP32 device offers a parallel audio-in/audio-out path. The graph also preserves grievance drafting, status tracking, history, analytics, and hardware monitoring. Some frontend and API subsystems are represented from documented routes and the source index; their internal behavior was not sampled.
Main entry points: main.py FastAPI entry · esp32_manager.py Audio processing · Chatbot.tsx Voice and text chat · rag_service.py RAG orchestration
Components 34
Kiosk experience
- App routes
src/App.tsx - Session state
src/contexts/AppContext.tsx - Voice and text chat
src/pages/Chatbot.tsx - Browser speech
src/hooks/useSpeech.ts - Offline answers
src/data/answers.ts - Language lookup
src/utils/lookup.ts
Citizen services
- Grievance workflow
src/pages/grievance - Grievance definitions
src/data/grievance.ts - Status tracking
src/pages/StatusTracker.tsx - Data overview
src/pages/DataOverview.tsx - Session history
src/data/history.ts
Backend API
- FastAPI entry
backend/app/main.py - Chat endpoint
backend/app/api/chat.py - Voice endpoint
backend/app/api/voice.py - Scheme endpoint
backend/app/api/schemes.py - Grievance endpoint
backend/app/api/grievances.py - Document endpoint
backend/app/api/documents.py
Answer intelligence
- RAG orchestration
backend/app/services/rag_service.py - Vector retrieval
backend/app/rag/retrieval/vector_search.py - Keyword retrieval
backend/app/rag/retrieval/keyword_search.py - Candidate reranking
backend/app/rag/retrieval/reranker.py - Query embeddings
backend/app/rag/embeddings/embedder.py - Grounded generation
backend/app/services/llm_service.py - Citation building
backend/app/rag/citations/citation_service.py - Policy data store
backend/app/db/database.py - Speech recognition
backend/app/services/stt_service.py - Speech synthesis
backend/app/services/tts_service.py
Voice hardware
- ESP32 device
- WebSocket manager
backend/app/hardware/websocket_manager.py - Device protocol
backend/app/hardware/protocol.py - Audio processing
backend/app/hardware/esp32_manager.py - Hardware status
src/components/status/HardwareStatusPanel.tsx
Other components
- Farmer / citizen
- Google Gemini
Connections 39
- Farmer / citizen → Voice and text chat: asks by voice/text
README.md - App routes → Voice and text chat: routes to
src/App.tsx - Voice and text chat → Session state: reads session
src/pages/Chatbot.tsx - Voice and text chat → Offline answers: uses answers
src/pages/Chatbot.tsx - Voice and text chat → Browser speech: uses speech
src/pages/Chatbot.tsx - Browser speech → Language lookup: looks up voice
src/hooks/useSpeech.ts - Voice and text chat → Chat endpoint: sends chat
src/pages/Chatbot.tsx - Voice and text chat → Voice endpoint: sends audio
src/pages/Chatbot.tsx - App routes → Grievance workflow: routes to
src/App.tsx - Grievance workflow → Grievance definitions: uses definitions
src/pages/grievance/GrievanceDraft.tsx - App routes → Status tracking: routes to
src/App.tsx - App routes → Data overview: routes to
src/App.tsx - Session state → Session history: loads history
src/contexts/AppContext.tsx - FastAPI entry → Chat endpoint: registers route
backend/app/main.py - FastAPI entry → Voice endpoint: registers route
backend/app/main.py - FastAPI entry → Scheme endpoint: registers route
backend/app/main.py - FastAPI entry → Grievance endpoint: registers route
backend/app/main.py - FastAPI entry → Document endpoint: registers route
backend/app/main.py - FastAPI entry → WebSocket manager: handles sockets
backend/app/main.py - Chat endpoint → RAG orchestration: retrieves context
backend/app/api/chat.py - Chat endpoint → Grounded generation: requests answer
backend/app/api/chat.py - RAG orchestration → Query embeddings: embeds query
backend/app/services/rag_service.py - RAG orchestration → Vector retrieval: searches vectors
backend/app/services/rag_service.py - RAG orchestration → Keyword retrieval: searches keywords
backend/app/services/rag_service.py - RAG orchestration → Candidate reranking: reranks candidates
backend/app/services/rag_service.py - Vector retrieval → Policy data store: reads vectors
backend/app/rag/retrieval/vector_search.py - Grounded generation → Google Gemini: generates with
README.md - FastAPI entry → Audio processing: dispatches audio
backend/app/main.py - Audio processing → Speech recognition: transcribes audio
backend/app/hardware/esp32_manager.py - Audio processing → RAG orchestration: retrieves context
backend/app/hardware/esp32_manager.py - Audio processing → Grounded generation: generates answer
backend/app/hardware/esp32_manager.py - Audio processing → Citation building: builds citations
backend/app/hardware/esp32_manager.py - Audio processing → Speech synthesis: synthesizes speech
backend/app/hardware/esp32_manager.py - Audio processing → WebSocket manager: streams response
backend/app/hardware/esp32_manager.py - WebSocket manager → Device protocol: uses states
backend/app/hardware/websocket_manager.py - ESP32 device → FastAPI entry: sends audio
backend/app/main.py - WebSocket manager → Hardware status: broadcasts status
README.md - Speech recognition → Language lookup: detects language
backend/app/services/stt_service.py - Speech synthesis → ESP32 device: returns audio
README.md