How satvik-gautam/ai-agentic-doctor works
Project Type and Purpose
• This is a modular, single‐page, full‐stack AI application built in Python with a web UI (Gradio) that simulates a virtual doctor consultation.
• It integrates speech‐to‐text, image analysis, and text‐to‐speech to process patient inputs and return medical insights.
High‐Level Components
A. Presentation Layer (Gradio UI)
• app.py: Defines the web interface, custom CSS, input widgets (microphone, file upload), and result panels.
B. Audio Ingestion & Transcription
• voice_of_the_patient.py: Captures microphone audio (SpeechRecognition + Pydub + PortAudio), saves MP3, calls Groq Whisper API for transcription.
C. Image Preprocessing & AI Core
• brain_of_doctor.py: Base64‐encodes uploaded images; invokes Groq Llama model with a system prompt combining text + optional image.
D. Response Synthesis & Playback
• voice_of_the_doctor.py: Takes AI’s text response, uses gTTS to generate MP3 (optionally ElevenLabs), returns audio file.
E. Configuration & Hosting
• .env + python-dotenv: Stores API keys for Groq and TTS services.
• .gradio/: Certificates and temporary hosted‐app files (Hugging Face Spaces integration).
F. External Services
• gTTS (Google Text‐to‐Speech)
• (Optional) ElevenLabs TTS
• Hugging Face Spaces (deployment environment)
Data Flow & Relationships
1) User clicks microphone → app.py routes to voice_of_the_patient.py → raw audio → Groq Whisper → transcription text → back to app.py
2) User uploads image (optional) → app.py hands file to brain_of_doctor.py → base64 encoding
3) app.py calls brain_of_doctor.py with text + image → Groq Llama → AI’s medical evaluation (text)
4) app.py sends AI text to voice_of_the_doctor.py → gTTS → MP3
5) app.py updates Gradio UI with: (a) transcription text, (b) AI’s text response, (c) auto-playing MP3.
Architectural Patterns & Principles
• Layered/Onion Architecture: Presentation → Orchestration (app.py) → Domain modules → External APIs.
• Separation of Concerns: Distinct modules for audio I/O, image handling, AI logic, and TTS.
• Modular & Extensible: Easy to switch TTS engines, add preprocessing steps, or change AI models.
• Environment‐Driven Configuration: All keys and endpoints in .env.