How dhrumilprajapati03/voice-chat-assistant works
Academy Assistant serves text and voice questions about courses, fees, schedules and academy documents. Users enter through the Gradio UI or FastAPI endpoints; the pipeline classifies and records text, handles greetings directly, and otherwise asks a session-aware agent. Voice requests add speech recognition before text handling and speech synthesis afterward. The agent can consult document retrieval and read-only academy SQL. History is stored separately from the agent’s in-memory conversational checkpoint. The source excerpts support the main runtime calls; ingestion details and database schema behavior are less fully shown.
Main entry points: pipeline.py Conversation pipeline · agent.py Agent orchestration · main.py FastAPI endpoints · db.py Chat history access
Components 17
User interaction
- Gradio chat UI
app/ui.py - FastAPI endpoints
app/main.py
Conversation runtime
- Conversation pipeline
app/pipeline.py - Intent classifier
tools/intent.py - Speech recognition
tools/stt.py - Speech synthesis
tools/tts.py - Agent orchestration
tools/agent.py
Knowledge access
- Document search tool
tools/rag.py - Chroma document index
tools/rag.py - Document ingestion
Scripts/ingest.py - Academy source documents
data - Academy SQL tool
tools/sql_tool.py
State and storage
- Chat history access
tools/db.py - Academy database
db/schema.sql - Agent session memory
tools/agent.py
Other components
- Academy user
- Groq language model
Connections 19
- Academy user → Gradio chat UI: uses
app/ui.py - Gradio chat UI → FastAPI endpoints: sends chat and voice
app/ui.py - Academy user → FastAPI endpoints: submits requests
app/main.py - FastAPI endpoints → Conversation pipeline: dispatches chat
app/main.py - FastAPI endpoints → Chat history access: loads history
app/main.py - Conversation pipeline → Intent classifier: classifies text
app/pipeline.py - Conversation pipeline → Chat history access: saves messages
app/pipeline.py - Conversation pipeline → Agent orchestration: asks agent
app/pipeline.py - Conversation pipeline → Speech recognition: transcribes audio
app/pipeline.py - Conversation pipeline → Speech synthesis: synthesizes reply
app/pipeline.py - Agent orchestration → Groq language model: generates replies
tools/agent.py - Agent orchestration → Document search tool: calls document search
tools/agent.py - Agent orchestration → Academy SQL tool: calls SQL tool
tools/agent.py - Document search tool → Chroma document index: searches documents
tools/rag.py - Document ingestion → Chroma document index: builds index
Scripts/ingest.py - Academy source documents → Document ingestion: provides sources
- Academy SQL tool → Academy database: queries
tools/sql_tool.py - Chat history access → Academy database: reads and writes
tools/db.py - Agent orchestration → Agent session memory: checkpoints sessions
tools/agent.py