# vyshnavnp/gemini-rag-chatbot architecture

> Architecture diagram of the GitHub repository vyshnavnp/gemini-rag-chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vyshnavnp/gemini-rag-chatbot
- Repository: https://github.com/vyshnavnp/gemini-rag-chatbot

## Overview

1. **Overall shape**
   - OncoBot is a **single-user-request, single-agent oncology assistant**. The runtime path is: **Streamlit UI → agent orchestrator → tools/cache/memory → response back to UI**.
   - It is not a multi-agent system; the core intelligence is one LangGraph agent that chooses among tools.

2. **User-facing boundary**
   - `app.py` is the entry point for the interactive UI.
   - It handles chat interactions and uploaded inputs, then hands the request to the agent layer.
   - For image/CSV uploads, the UI is only the ingress point; the data is not routed as large tool arguments through the LLM.

3. **Agent orchestration layer**
   - `agent/onco_agent.py` contains the central LangGraph flow.
   - The agent uses **Gemini 3.1 Flash Lite Preview** as the reasoning model and runs a **ReAct-style loop** with up to 5 tool iterations per turn.
   - The agent is the policy boundary: it decides whether to answer from knowledge retrieval, external search, vision, or ML classification.

4. **Semantic cache first**
   - `agent/cache.py` implements a **ChromaDB-backed response cache**.
   - Every query is checked before the LLM is called; if cosine similarity is high enough, the system returns immediately and avoids API usage.
   - This cache is a major architectural boundary because it can short-circuit the entire agent/tool pipeline.

5. **Conversational memory**
   - `agent/memory.py` provides session-thread memory via LangGraph checkpointing.
   - This lets follow-up questions work without re-supplying prior context.
   - Memory is separate from the semantic cache: cache is for repeat/paraphrased queries, memory is for in-session continuity.

6. **Tool subsystem: internal knowledge and retrieval**
   - `tools/onco_tools.py` holds the main oncology tools.
   - The key retrieval path is **RAG over the local knowledge base** stored in `knowledge_base/`, which contains MedQuAD-style XML and some PDF documents.
   - The embeddings are multilingual, so non-English queries can still match English source material.

7. **Tool subsystem: live external research**
   - `tools/external_tools.py` exposes live research tools against external services:
     - **PubMed / NCBI E-utilities**
     - **ClinicalTrials.gov v2**
     - **arXiv**
   - These tools make the assistant current, but they are separate from the local knowledge-base RAG path.

8. **Tool subsystem: vision and medical image analysis**
   - The app supports multimodal input for images.
   - Uploaded image bytes are stored in **session-level shared state** inside the tool layer, while the image is also passed to Gemini as multimodal content.
   - This allows the agent to either reason over the image with Gemini Vision or route to a specialized classifier.

9. **Tool subsystem: ML classifiers**
   - The repo ships local models under `models/` for three classification tasks:
     - **Breast ultrasound** classification
     - **Skin lesion** classification
     - **Cancer type from gene expression**
   - These models are loaded lazily and run locally on CPU with PyTorch; they are not called through external services.
   - The gene-expression classifier also depends on preprocessing artifacts: a scaler and label encoder.

10. **Data and state flow for uploads**
    - `app.py` reads uploaded files, `agent/onco_agent.py` stores them in session state, and tools read them directly from shared memory.
    - This avoids pushing raw binaries through the LLM tool-call interface.
    - After the agent finishes, session upload state is cleared so the next turn starts cleanly.

11. **Knowledge-base maintenance**
    - `updater.py` maintains the retrieval corpus by re-indexing `knowledge_base/`.
    - The readme indicates incremental updates use SHA-256 hashing and periodic refresh via APScheduler, so the retrieval store stays in sync without duplicating unchanged documents.
    - This is an offline/maintenance path separate from normal user queries.

12. **Evaluation path**
    - `evaluation/ragas_eval.py` is not part of serving traffic.
    - It measures answer quality with an LLM-as-judge approach, mainly for faithfulness and relevance.
    - Architecturally, this is a sidecar QA loop around the retrieval/agent system, not a runtime dependency.

13. **Persistence and packaging**
    - ChromaDB provides the vector/cached state; model weights and lookup artifacts live locally in the repo.
    - The application is containerized with **Docker** and optionally run via **Docker Compose**.
    - This makes the repo self-contained for local execution and EC2 deployment.

14. **Deployment and operational boundary**
    - CI/CD is defined in `.github/workflows/deploy.yml`.
    - The deployment path is: **GitHub push → build Docker image → security scan → push to ECR → deploy to AWS EC2**.
    - The runtime is a **Streamlit app in a Linux container**, with security hardening such as non-root execution and restricted capabilities.

15. **How to draw the diagram**
    - The main boxes are: **UI**, **semantic cache**, **agent orchestrator**, **tool layer**, **local knowledge base/vector store**, **external research APIs**, **local ML models**, and **memory**.
    - The most important arrows are:
      - UI → cache → agent
      - agent ↔ tools
      - tools ↔ ChromaDB/knowledge base
      - tools ↔ external APIs
      - tools ↔ local PyTorch models
      - agent ↔ memory
      - uploads → session state → image/CSV tools

Main entry points: [`tools/onco_tools.py`](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/tools/onco_tools.py) (Tool Hub), [`app.py`](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/app.py) (Streamlit UI), [`agent/onco_agent.py`](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/onco_agent.py) (Onco Agent), [`agent/cache.py`](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/cache.py) (Semantic Cache)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui_runtime["UI Runtime"]
  node_app["Streamlit UI<br/>entrypoint<br/>[app.py]"]
end

subgraph group_agent_layer["Agent Layer"]
  node_cache[("Semantic Cache<br/>Chroma cache<br/>[cache.py]")]
  node_agent{{"Onco Agent<br/>LangGraph orchestrator<br/>[onco_agent.py]"}}
  node_memory[("Session Memory<br/>checkpoint memory<br/>[memory.py]")]
end

subgraph group_data_tools["Data and Tools"]
  node_toolkit["Tool Hub<br/>tool router<br/>[onco_tools.py]"]
  node_research["Research APIs<br/>external tools<br/>[external_tools.py]"]
  node_kb_rag["KB RAG<br/>retrieval pipeline<br/>[onco_tools.py]"]
  node_upload_state["Upload State<br/>session shared state<br/>[onco_tools.py]"]
  node_vision(("Vision Path<br/>multimodal reasoning<br/>[onco_agent.py]"))
  node_bc_model["Breast Model<br/>PyTorch classifier<br/>[oncoscan_bc.pth]"]
  node_skin_model["Skin Model<br/>PyTorch classifier<br/>[oncoscan_skin.pth]"]
  node_gene_model["Gene Model<br/>PyTorch classifier<br/>[oncotype_bc.pth]"]
  node_preproc["Preprocess Artifacts<br/>scaler and labels<br/>[scaler.pkl]"]
  node_knowledge_base[("Knowledge Base<br/>local corpus")]
end

subgraph group_ops_quality["Ops and Quality"]
  node_updater["Corpus Updater<br/>index maintenance<br/>[updater.py]"]
  node_eval["RAG Eval<br/>quality evaluation<br/>[ragas_eval.py]"]
  node_docker["Container Deploy<br/>build and deploy"]
end

node_app -->|"check query"| node_cache
node_cache -->|"miss to agent"| node_agent
node_cache -->|"hit return"| node_app
node_app -->|"submit turn"| node_agent
node_agent -->|"read/write"| node_memory
node_agent -->|"select tools"| node_toolkit
node_toolkit -->|"retrieve"| node_kb_rag
node_kb_rag -->|"read corpus"| node_knowledge_base
node_toolkit -->|"live search"| node_research
node_toolkit -->|"read uploads"| node_upload_state
node_app -->|"store files"| node_upload_state
node_toolkit -->|"analyze image"| node_vision
node_vision -->|"use image bytes"| node_upload_state
node_toolkit -->|"classify"| node_bc_model
node_toolkit -->|"classify"| node_skin_model
node_toolkit -->|"classify"| node_gene_model
node_gene_model -->|"preprocess"| node_preproc
node_updater -->|"reindex"| node_knowledge_base
node_eval -.->|"score outputs"| node_kb_rag
node_docker -->|"serve UI"| node_app
node_docker -->|"package runtime"| node_agent

click node_app "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/app.py"
click node_cache "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/cache.py"
click node_agent "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/onco_agent.py"
click node_memory "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/memory.py"
click node_toolkit "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/tools/onco_tools.py"
click node_research "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/tools/external_tools.py"
click node_kb_rag "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/tools/onco_tools.py"
click node_upload_state "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/tools/onco_tools.py"
click node_vision "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/onco_agent.py"
click node_bc_model "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/models/oncoscan_bc.pth"
click node_skin_model "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/models/oncoscan_skin.pth"
click node_gene_model "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/models/oncotype_bc.pth"
click node_preproc "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/models/scaler.pkl"
click node_knowledge_base "https://github.com/vyshnavnp/gemini-rag-chatbot/tree/main/knowledge_base"
click node_updater "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/updater.py"
click node_eval "https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/evaluation/ragas_eval.py"
click node_docker "https://github.com/vyshnavnp/gemini-rag-chatbot/tree/main/Dockerfile"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app toneBlue
class node_cache,node_agent,node_memory toneAmber
class node_toolkit,node_research,node_kb_rag,node_upload_state,node_vision,node_bc_model,node_skin_model,node_gene_model,node_preproc,node_knowledge_base toneMint
class node_updater,node_eval,node_docker toneRose
```

## Components

### UI Runtime

Interactive request ingress and serving path

- [Streamlit UI](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/app.py): entrypoint · Interactive chat and upload ingress · `app.py`

### Agent Layer

Orchestration, cache, memory, and tool routing

- [Semantic Cache](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/cache.py): Chroma cache · Short-circuits repeat and paraphrased queries before LLM use · `agent/cache.py`
- [Onco Agent](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/onco_agent.py): LangGraph orchestrator · Single-agent ReAct loop that selects between retrieval, search, vision, and classifiers · `agent/onco_agent.py`
- [Session Memory](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/memory.py): checkpoint memory · LangGraph checkpointed conversational state for follow-ups · `agent/memory.py`

### Data and Tools

Retrieval corpus, live research, multimodal inputs, and local models

- [Tool Hub](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/tools/onco_tools.py): tool router · Main oncology tool surface invoked by the agent · `tools/onco_tools.py`
- [Research APIs](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/tools/external_tools.py): external tools · Live PubMed, ClinicalTrials.gov, and arXiv lookups · `tools/external_tools.py`
- [KB RAG](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/tools/onco_tools.py): retrieval pipeline · Local knowledge-base retrieval over MedQuAD-style XML and PDFs · `tools/onco_tools.py`
- [Upload State](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/tools/onco_tools.py): session shared state · Holds uploaded image and CSV bytes across the turn · `tools/onco_tools.py`
- [Vision Path](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/agent/onco_agent.py): multimodal reasoning · Gemini-assisted image reasoning over uploaded medical images · `agent/onco_agent.py`
- [Breast Model](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/models/oncoscan_bc.pth): PyTorch classifier · Local breast ultrasound classifier loaded on demand · `models/oncoscan_bc.pth`
- [Skin Model](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/models/oncoscan_skin.pth): PyTorch classifier · Local skin lesion classifier loaded on demand · `models/oncoscan_skin.pth`
- [Gene Model](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/models/oncotype_bc.pth): PyTorch classifier · Local cancer-type classifier from gene expression · `models/oncotype_bc.pth`
- [Preprocess Artifacts](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/models/scaler.pkl): scaler and labels · Scaler and label encoder used by the gene-expression model · `models/scaler.pkl`
- [Knowledge Base](https://github.com/vyshnavnp/gemini-rag-chatbot/tree/main/knowledge_base): local corpus · XML and PDF source corpus used for retrieval and reindexing · `knowledge_base`

### Ops and Quality

Maintenance, evaluation, and deployment

- [Corpus Updater](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/updater.py): index maintenance · Re-indexes the corpus with incremental hashing and periodic refresh · `updater.py`
- [RAG Eval](https://github.com/vyshnavnp/gemini-rag-chatbot/blob/main/evaluation/ragas_eval.py): quality evaluation · LLM-as-judge evaluation for faithfulness and relevance · `evaluation/ragas_eval.py`
- [Container Deploy](https://github.com/vyshnavnp/gemini-rag-chatbot/tree/main/Dockerfile): build and deploy · Docker and GitHub Actions path for container build, scan, and EC2 deploy · `Dockerfile`

## Connections

- Streamlit UI → Semantic Cache: check query · Incoming request is checked against semantic cache first
- Semantic Cache → Onco Agent: miss to agent · Uncached queries proceed to the orchestrator
- Semantic Cache → Streamlit UI: hit return · High-similarity matches return directly to the UI
- Streamlit UI → Onco Agent: submit turn · User text and uploads enter the agent boundary
- Onco Agent → Session Memory: read/write · Conversation state is loaded and checkpointed per turn
- Onco Agent → Tool Hub: select tools · ReAct loop invokes tool calls through the main tool hub
- Tool Hub → KB RAG: retrieve · Local retrieval over the knowledge-base corpus
- KB RAG → Knowledge Base: read corpus · Retrieval operates on the local XML and PDF source set
- Tool Hub → Research APIs: live search · External research is routed to live biomedical and literature APIs
- Tool Hub → Upload State: read uploads · Tools consume shared session bytes instead of raw LLM arguments
- Streamlit UI → Upload State: store files · Uploaded images and CSVs are staged in session state
- Tool Hub → Vision Path: analyze image · Image reasoning uses multimodal model support when needed
- Vision Path → Upload State: use image bytes · Vision path consumes stored uploaded bytes
- Tool Hub → Breast Model: classify · Breast ultrasound inference runs locally on CPU
- Tool Hub → Skin Model: classify · Skin lesion inference runs locally on CPU
- Tool Hub → Gene Model: classify · Gene-expression cancer typing runs locally on CPU
- Gene Model → Preprocess Artifacts: preprocess · Model inference depends on scaler and label encoder artifacts
- Corpus Updater → Knowledge Base: reindex · Offline maintenance refreshes the retrieval corpus
- RAG Eval → KB RAG: score outputs · Evaluation wraps the retrieval and answer generation quality
- Container Deploy → Streamlit UI: serve UI · Container build packages the Streamlit serving runtime
- Container Deploy → Onco Agent: package runtime · Deployment ships the agent stack inside the container
