How foxder15/health-chatbot works
Project type and purpose
• This is a full-stack Health Chatbot application.
• Purpose: Accept health-related queries via a Streamlit UI, dispatch them to a FastAPI backend, run a Retrieval-Augmented Generation (RAG) pipeline (using crewAI, custom embedding stores, and document preprocessing), and return generated answers.
File structure
• Top-level:
• app.py (Streamlit frontend entry point)
• src/apis/query_api.py (FastAPI backend entry point)
• src/RAG – modules for advanced RAG logic, embedding modification, preprocessing
• src/embedding – per-scenario vector stores and JSON doc/graph stores
• src/config – YAML definitions of agents and tasks
• crew.py – orchestration script
• db/ & src/default.sqlite / db/chroma.sqlite3 – persistent data stores
• document/ & public/ – sample corpora, chat histories, evaluation artifacts
• evaluation.ipynb & report.md – offline evaluation and reporting
• requirements.txt – dependencies (FastAPI, Streamlit, crewAI, Chroma, etc.)
README
• Confirms FastAPI for backend, Streamlit for frontend, crewAI for LLM orchestration
• Instructions to launch two processes:
• Uvicorn → src/apis/query_api:app
• streamlit run app.py
Diagramming the architecture
a. Main components
1. Frontend: Streamlit UI (app.py)
2. Backend: FastAPI Query API (query_api:app)
3. RAG Pipeline:
– Preprocessing module
– Embedding modules (src/embedding/…)
– Retrieval engine (Chroma DB / vector stores)
– LLM orchestrator (crewAI via crew.py)
4. Data Stores:
– Binary/document store in db/ and src/embedding/*
– SQLite databases (chroma.sqlite3, default.sqlite)
5. Configuration: YAML files for agents/tasks
6. External Services:
– crewAI LLM endpoint (cloud or local)
7. Evaluation: Jupyter notebook + report for offline testing
b. Relationships & interactions
• User → (HTTP over localhost:8501) → Streamlit UI
• Streamlit → (HTTP REST) → FastAPI Query API
• Query API → invokes RAG modules in-process
• Preprocessing → text & PDF loader → normalized docs
• Embedding module → produces vectors → stored/read from Chroma & JSON stores
• Retrieval → top-k matches → fed to LLM orchestrator
• LLM orchestrator (crew.py) → crewAI API → generates response
• Response flows back through FastAPI → Streamlit → User
• Evaluation pipeline reads same stores, runs offline tests
c. Architectural patterns & principles
• Separation of concerns: UI vs API vs RAG logic vs storage
• Modular RAG pipeline: preprocessing, embedding, retrieval stages
• Configuration-driven behavior: agents.yaml & tasks.yaml
• Persistence layer abstracted via vector stores and SQLite
d. Key technologies & libraries
• Frontend: Streamlit
• Backend: FastAPI, Uvicorn
• RAG: custom Python modules, Chroma vector DB
• Orchestration: crewAI, crew.py
• Data formats: .bin, .json, SQLite
• Configuration: YAML