How utkarsh-chauhannn/voice-assistant-doctor works
Project type and purpose
• This is a Python-based, full-stack, voice-enabled “AI Doctor” assistant. It uses Gradio as the user interface, custom Python modules for speech-to-text (STT), an LLM-based “brain” for generating responses, and text-to-speech (TTS) engines to speak back to the user. It is containerized with Docker.
File structure
• Top-level files:
• gradio_app.py – entry point for the Gradio UI (frontend)
• brain_of_the_doctor.py – core logic invoking the LLM
• voice_of_the_patient.py – speech recognition / STT wrapper
• voice_of_the_doctor.py – TTS wrapper (ElevenLabs, gTTS)
• requirements.txt / Pipfile – dependency specs
• Dockerfile – container orchestration
• .gradio/ directory holds session data, flagged inputs, and uploaded media.
• Assets: several .mp3 test files show usage of ElevenLabs and gTTS.
README
• Title “Voice-Assistant-Doctor” / “AI-Doctor” confirms the purpose: a conversational doctor assistant that takes voice input, reasons with AI, and answers with a synthesized voice.
Main components and interactions (to include
a. Frontend (Gradio UI)
– gradio_app.py serves a web interface where patients “Speak Your Symptoms” or upload an image/audio.
b. Audio Ingestion & STT
– voice_of_the_patient.py listens to uploaded or live audio, calls a speech-to-text engine (e.g., OpenAI Whisper or a third-party API), produces a text transcript.
c. AI Brain
– brain_of_the_doctor.py receives the transcript, formats a prompt, calls the OpenAI (or similar) API, and gets a text response.
d. TTS
– voice_of_the_doctor.py feeds the text response to ElevenLabs or Google TTS, producing an audio file.
e. Data Storage & Logging
– .gradio/ holds session files, flagged content, and CSV logs (dataset1.csv, dataset2.csv).
f. External Services
– OpenAI API (LLM calls)
– ElevenLabs / gTTS APIs (TTS)
g. Deployment
– Docker container built from Dockerfile, running all modules in a single service.
Architectural patterns & design principles
• Pipeline pattern: clear separation of STT → NLP → TTS.
• Modular design: each concern lives in its own Python module.
• Stateless UI: Gradio serves requests, all state logged to disk.
• External-service integration: LLM and TTS engines are treated as downstream services.
Detail is key
• Every Python script as its own component.
• .gradio/ as a persistent storage block.
• Configuration files (Pipfile, requirements.txt, Dockerfile) in a “Deployment & Config” section.