How entbappy/end-to-end-medical-chatbot-using-llama2 works
Project type and purpose
• This is a full-stack, retrieval-augmented medical chatbot built with Python, Flask and LangChain.
• It ingests a medical PDF, indexes it in Pinecone, and serves chat queries to a local Llama 2 model via a Flask web UI.
File structure
Top-level files
• app.py – Flask application exposing REST endpoints for chat.
• store_index.py – ingestion pipeline that reads data/Medical_book.pdf, creates embeddings, and writes them to Pinecone.
• template.py – prompt‐template definitions.
• requirements.txt / setup.py – dependency and install scripts.
• .gitignore, LICENSE, README.md – project metadata.
Data and model
• data/Medical_book.pdf – the core knowledge source.
• model/instruction.txt – custom system prompt instructions for Llama 2.
Source code
• src/helper.py – utility functions (e.g., PDF loader, embedding helper).
• src/prompt.py – constructs LangChain prompt patterns.
• static/style.css & templates/chat.html – front-end UI assets.
Research
• research/trials.ipynb – exploratory notebooks.
Key README insights
• Uses LangChain for RAG.
• Flask for web server.
• Pinecone for vector store.
• Meta Llama2 (quantized .bin) as the local LLM.
Main components and interactions
a. Frontend (Browser)
• chat.html + style.css – user interface to send queries and show responses.
– communicates via HTTP(S) to Flask backend.
b. Backend (Flask app.py)
• REST endpoint (/chat or root)
• Loads environment variables (.env) for Pinecone credentials.
• Uses helper functions to:
– Submit user query
– Retrieve context from Pinecone
– Construct prompt via template.py / src/prompt.py
– Call the local Llama 2 model (llama_cpp or similar) with model/instruction.txt
c. Vector Store (Pinecone)
• Stores document embeddings for Medical_book.pdf (created by store_index.py).
• Backend queries nearest neighbors given a new query embedding.
d. Ingestion Pipeline (store_index.py)
• Loads PDF from data/
• Splits into chunks, computes embeddings (via same embedding model), pushes to Pinecone.
e. Local LLM (Llama 2 .bin)
• Quantized model loaded by LangChain’s LLM wrapper.
• Receives system + user + context prompt and returns a response.
Architectural patterns and design principles
• Separation of concerns: ingestion vs. serving pipeline.
• Retrieval-augmented generation (RAG) pattern.
• Modular prompt engineering (template.py + src/prompt.py).
• Single-process Flask server suitable for prototyping, can evolve into microservices.
Technologies & frameworks to annotate
• Flask
• LangChain
• Pinecone
• Llama2 (ggml quantized)
• Conda environment for reproducibility
• HTML/CSS for frontend