How souvikmajumder26/multi-agent-medical-assistant works
This is a multi-agent medical assistant: users submit text or medical images through the API, and an orchestration graph applies guardrails and routes work to conversational, RAG, web-research, or imaging agents. RAG supports document ingestion, parsing, chunking, query expansion, retrieval, reranking, and grounded response generation; low-confidence answers can hand off to web search. Image results may require human validation, and the API also supports speech input/output. Relationships reflect the supplied excerpts, source index, and README; unsampled implementation details are represented only where documented.
Main entry points: __init__.py Medical RAG · agent_decision.py Agent Router · image_classifier.py Image Classifier · app.py HTTP API
Components 25
User Interface
- HTTP API
app.py - Speech Services
app.py
Agent Orchestration
- Agent Router
agents/agent_decision.py - Input Guardrails
agents/guardrails/local_guardrails.py - Agent Configuration
config.py - Human Validation
app.py
Medical Knowledge
- Medical RAG
agents/rag_agent/__init__.py - Document Parser
agents/rag_agent/doc_parser.py - Content Processor
agents/rag_agent/content_processor.py - Query Expansion
agents/rag_agent/query_expander.py - Hybrid Retrieval
agents/rag_agent/vectorstore_qdrant.py - Qdrant Store
data/qdrant_db - Chunk Reranker
agents/rag_agent/reranker.py - RAG Response
agents/rag_agent/response_generator.py - Document Ingestion
ingest_rag_data.py
Research Search
- Web Search Processor
agents/web_search_processor_agent/web_search_processor.py - Web Search Agent
agents/web_search_processor_agent/web_search_agent.py - PubMed Search
agents/web_search_processor_agent/pubmed_search.py - Tavily Search
agents/web_search_processor_agent/tavily_search.py
Image Analysis
- Image Classifier
agents/image_analysis_agent/image_classifier.py - Chest X-ray Model
agents/image_analysis_agent/chest_xray_agent/covid_chest_xray_inference.py - Skin Lesion Model
agents/image_analysis_agent/skin_lesion_agent/skin_lesion_inference.py - Brain Tumor Model
agents/image_analysis_agent/brain_tumor_agent/brain_tumor_inference.py
Other components
- User
- ElevenLabs
Connections 24
- User → HTTP API: submits query
README.md - HTTP API → Agent Router: routes chat
app.py - HTTP API → Speech Services: handles audio
app.py - Speech Services → ElevenLabs: uses speech API
app.py - Agent Router → Input Guardrails: checks input
agents/agent_decision.py - Agent Router → Medical RAG: routes queries
agents/agent_decision.py - Agent Router → Web Search Processor: hands off
agents/agent_decision.py - Agent Router → Image Classifier: routes images
agents/agent_decision.py - Agent Router → Human Validation: requests review
README.md - Medical RAG → Document Parser: parses documents
agents/rag_agent/__init__.py - Medical RAG → Content Processor: processes content
agents/rag_agent/__init__.py - Medical RAG → Query Expansion: expands queries
agents/rag_agent/__init__.py - Medical RAG → Hybrid Retrieval: retrieves chunks
agents/rag_agent/__init__.py - Medical RAG → Chunk Reranker: reranks results
agents/rag_agent/__init__.py - Medical RAG → RAG Response: generates answer
agents/rag_agent/__init__.py - Hybrid Retrieval → Qdrant Store: reads and writes
README.md - Document Ingestion → Medical RAG: calls ingestion
ingest_rag_data.py - Web Search Processor → Web Search Agent: searches query
agents/web_search_processor_agent/web_search_processor.py - Web Search Agent → PubMed Search: uses search
agents/web_search_processor_agent/web_search_agent.py - Web Search Agent → Tavily Search: uses search
agents/web_search_processor_agent/web_search_agent.py - Image Classifier → Chest X-ray Model: dispatches analysis
agents/image_analysis_agent/__init__.py - Image Classifier → Skin Lesion Model: dispatches analysis
agents/image_analysis_agent/__init__.py - Image Classifier → Brain Tumor Model: dispatches analysis
- Agent Configuration → Medical RAG: configures agents