How sunbird-aiassistant/sakhi-api-service works
Project Type & Purpose
• This is a Python-based backend API service (FastAPI) exposing two endpoints: POST /v1/query and POST /v1/chat.
• It ingests user text/audio queries, invokes LLMs (Azure/OpenAI/Ollama), fetches context from a vector store (Marqo), performs translation and TTS/STT, caches chats in Redis, stores audio in cloud storage, and logs telemetry.
High-Level File Structure Breakdown
• main.py + uvicorn entrypoint → FastAPI app & routers
• env_manager.py, config.ini, .env → configuration & environment
• llm/ → adapters for various LLM providers (azure_openai.py, openai.py, ollama.py)
• vectorstores/ → Marqo integration
• translation/ → text↔text and speech↔text modules (Google, Bhashini, Dhruva, TTS/STT)
• storage/ → cloud object storage adapters (AWS, GCP, OCI) for audio files
• redis_util.py → Redis caching for /v1/chat
• telemetry_logger.py, telemetry_middleware.py → telemetry event emission
• index_documents.py → data ingestion into Marqo
• utils/ → common utilities (env loading, helpers)
Core Components
1. API Layer (FastAPI)
– Routers for /v1/query and /v1/chat
– Request parsing, validation
– Telemetry middleware and general logging
2. Business Orchestrator
– QueryController (in main or a separate module) that sequences translation, LLM calls, vector retrieval, and post-processing
3. Translation Service
– Text translation submodule (Google, Bhashini, Dhruva)
– Speech-to-Text (audio→text)
– Text-to-Speech (text→audio)
4. LLM Service Layer
– Base adapter → azure_openai, openai, ollama
– Configurable prompt injection (activity_prompt, chat_intent_prompt, etc.)
5. Vector Store
– Marqo index for document retrieval
– Configured via Docker container (8882)
6. Caching
– Redis (ttl configurable)
– Used only for /v1/chat conversation history
7. Cloud Storage
– AWS/GCP/OCI bucket adapters
– Public audio URL generation for TTS outputs
8. Telemetry & Logging
– Sunbird telemetry events (batching via telemetry_logger)
– Application logs via logger.py
External Dependencies
• Marqo vector index (Docker container)
• Redis instance
• Azure/OpenAI/Ollama API endpoints
• Cloud object storage (AWS S3, GCP Cloud Storage, OCI Object Storage)
• Optional: Google Translation API, Bhashini/Dhruva translation services
Architectural Patterns & Principles
• Modular adapters (LLM, Storage, Translation) — easy to swap providers
• Orchestrator (choreography) with clear sequencing
• Separation of concerns: routing / orchestration / external calls / utilities
• Config-driven behavior via config.ini/.env
• Caching for chat history to improve context retrieval
• Async calls (FastAPI + uvicorn) for scalability
Final Checklist for the Engineer
– Break down the system into the 8 core components listed above
– Capture the two API flows (/v1/query vs. /v1/chat) side by side or in a single merged flow
– Highlight modular adapters and where you could plug in new providers