How kiendoo4/doctorqa works
Project Type & Purpose
• DoctorQA is an AI-powered medical Q&A web application targeted at Vietnamese users. It integrates advanced AI techniques, including Chain-of-Thought (CoT) reasoning and Retrieval-Augmented Generation (RAG) through the LangChain framework and an LLM (Gemini-1.5). It also uses a Qdrant vector database for medical document embeddings and PostgreSQL for user and conversation management.
Analyzing the File Structure
• Key Backend Files: Files like main.py, database.py, CoT.py, and CoT_ver2.py indicate server-side logic, AI reasoning, and database interactions.
• Frontend Components: The static and templates directories house user interface assets (CSS, JavaScript, HTML) for the chat UI, registration, and the home/index page.
• Data Processing: The Jupyter Notebook (crawled-data-for-doctorqa.ipynb) indicates a separate data ingestion and processing pipeline used to crawl, clean, and vectorize over 30,000 medical documents.
• Additional Tools: test_model.py suggests some form of testing or validation, likely for the model integrations.
Main Components to Capture
a. Frontend UI:
– Web pages rendered from the /templates directory (index.html, chatUI.html, registration.html)
b. Backend API & Web Server:
– Flask-based server (main.py) that handles HTTP requests.
– Specific endpoints for login, registration (e.g., static/registration.py may be part of the workflow), and chat interactions.
c. AI Components:
– Integration with the LLM (Gemini-1.5) for generating context-aware answers.
– Chain-of-Thought logic modules (CoT.py and CoT_ver2.py) to enable enhanced reasoning in responses.
– LangChain as the orchestrator coordinating between LLM and data retrieval.
d. Data Storage:
– Qdrant Vector Database: Used to store and query vectorized medical documents. This serves the retrieval segment for RAG.
– PostgreSQL: Structured storage for user management (sessions, login data) and conversation logging.
e. Data Ingestion Pipeline:
– Data crawling and processing (via crawled-data-for-doctorqa.ipynb) that feeds data into Qdrant.
f. External & Auxiliary Components:
– Any error or fallback pages (e.g., a "failed.html" under the next_page directory) for handling unsuccessful operations.
Relationships and Data Flow
• Frontend UI communicates with the backend API via HTTP calls (AJAX or form submissions), which is served by Flask.
• The backend invokes the LangChain framework for coordinating with the Gemini-1.5 LLM and the CoT modules, ensuring that the questions get processed with enhanced reasoning.
• For medical knowledge retrieval, the backend queries the Qdrant vector database with the processed query to fetch relevant documents.
• User-related and logging operations are handled by PostgreSQL.
• The Data Ingestion component (from the Jupyter Notebook) periodically updates or feeds new data into Qdrant.
Architectural Patterns & Design Principles
• Layered architecture separating frontend, backend business logic, data management, and AI orchestration.
• Service orchestration using LangChain that encapsulates interactions between the LLM, chain-of-thought processing, and the data layer.
• Emphasis on modularity where each component (UI, API, AI modules, databases) is logically isolated, making it easier to update or scale parts independently.