# srujanrana07/policy-rag-v2 architecture

> Architecture diagram of the GitHub repository srujanrana07/policy-rag-v2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/srujanrana07/policy-rag-v2
- Repository: https://github.com/srujanrana07/policy-rag-v2

## Overview

1. **What this repository is**
   - A single-service RAG application for answering questions over insurance policy PDFs.
   - The system is built around a FastAPI backend that orchestrates PDF ingestion, retrieval, and Gemini-based answer generation.

2. **Primary runtime boundary**
   - The core application lives in `app/` and is deployed as one container.
   - `app/main.py` is the orchestration layer: it exposes HTTP endpoints, manages lifecycle setup, and holds the in-memory document cache.
   - `run.py` is the local dev entrypoint; Docker/Cloud Run is the production runtime.

3. **Document ingestion pipeline**
   - `document_loader.py` handles remote PDF retrieval and parsing.
   - Text extraction uses PyMuPDF, with table extraction via Camelot lattice mode and pdfplumber as fallback.
   - This stage turns one or more PDF URLs into structured text/table content ready for chunking.

4. **Chunking and embedding**
   - `vectorizer.py` is responsible for semantic chunking and building embeddings.
   - The embedding model is Sentence-Transformers `all-MiniLM-L6-v2`, exported to ONNX for CPU inference.
   - Output from this stage becomes the basis for similarity search and FAISS indexing.

5. **Search index and cache**
   - Each unique set of document URLs maps to an in-memory FAISS index.
   - `main.py` keeps an LRU cache of these document sets so repeated questions over the same PDFs skip download, extraction, and embedding.
   - The cache is process-local and intentionally ephemeral, matching Cloud Run’s stateless container model.

6. **Retrieval layer**
   - `retriever.py` implements the document search stack.
   - The README describes hybrid retrieval: semantic search from FAISS, lexical BM25, and a custom reranker.
   - Query expansion is part of this layer too, using abbreviation/synonym handling for insurance terms like PED, ICU, SI, and NCB.

7. **LLM answer generation**
   - `llm_client.py` wraps Google Gemini 2.5 Flash.
   - It supports both synchronous question answering and token-streaming output for the SSE endpoint.
   - Retrieval results are passed to Gemini as grounded context for final answer synthesis.

8. **API surface**
   - The backend exposes a small set of endpoints: health, synchronous multi-question query, streaming query, cache inspection, and cache reset.
   - `POST /query` is the batch Q&A path.
   - `GET /stream-query` is the interactive path, returning server-sent events token by token.

9. **UI boundary**
   - `static/index.html` is a single-file frontend with no separate build system.
   - It is a lightweight client for submitting document URLs/questions and consuming either JSON responses or SSE streams.
   - The README notes that users point the UI at the deployed Cloud Run service URL.

10. **Cold-start and model packaging**
   - The Docker image bakes in the embedding model in ONNX form.
   - This avoids PyTorch at runtime and prevents network downloads for model weights during cold start.
   - The architecture is optimized for CPU-only inference on Cloud Run.

11. **Deployment and CI/CD**
   - `cloudbuild.yaml` supports automated build-and-deploy from source.
   - `deploy-cloudrun.sh` is the manual path that enables APIs, builds the container, and deploys to Cloud Run.
   - Secret handling is externalized through Google Secret Manager for the Gemini API key.

12. **Operational model**
   - The service is designed to scale horizontally through Cloud Run instances, not multiple workers in one process.
   - The README explicitly notes `workers 1` because the FAISS index and LRU cache live in memory inside a single process.
   - `min-instances=0` keeps the service cost-efficient when idle.

13. **End-to-end data flow**
   - Input: one or more PDF URLs plus a question or set of questions.
   - Processing: download → extract text/tables → chunk → embed → index/cache.
   - Retrieval: embed question → hybrid search/rerank → provide evidence context.
   - Generation: Gemini produces the answer, either as a JSON response or streamed SSE tokens.

14. **Key architectural constraint**
   - The central tradeoff is speed versus persistence: caching and FAISS live in memory for fast repeated access, but nothing survives container restarts.
   - That makes the service simple and Cloud Run-friendly, but also means every fresh instance rebuilds document indices on demand.

Main entry points: [`app/main.py`](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py) (API app), [`app/retriever.py`](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/retriever.py) (Retriever), [`static/index.html`](https://github.com/srujanrana07/policy-rag-v2/blob/main/static/index.html) (Web client), [`deploy-cloudrun.sh`](https://github.com/srujanrana07/policy-rag-v2/blob/main/deploy-cloudrun.sh) (Deploy script)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Application runtime"]
  node_run_py["run.py<br/>dev entrypoint<br/>[run.py]"]
  node_main["API app<br/>FastAPI orchestrator<br/>[main.py]"]
  node_loader["Loader<br/>PDF ingest<br/>[document_loader.py]"]
  node_vectorizer["Vectorizer<br/>chunk+embed<br/>[vectorizer.py]"]
  node_retriever["Retriever<br/>hybrid search<br/>[retriever.py]"]
  node_llm["Gemini client<br/>LLM wrapper<br/>[llm_client.py]"]
  node_faiss_cache[("FAISS cache<br/>in-memory index")]
  node_pdf_source["PDF URLs<br/>external input"]
  node_answer_flow{{"Answer flow<br/>request pipeline"}}
  node_health_api(("Health<br/>endpoint<br/>[main.py]"))
  node_query_api(("Batch query<br/>endpoint<br/>[main.py]"))
  node_stream_api(("Stream query<br/>SSE endpoint<br/>[main.py]"))
  node_cache_api(("Cache ops<br/>endpoint<br/>[main.py]"))
end

subgraph group_ui["User interface"]
  node_static_ui["Web client<br/>static frontend<br/>[index.html]"]
end

subgraph group_deploy["Deployment"]
  node_dockerfile["Dockerfile<br/>container build"]
  node_cloudbuild["Cloud Build<br/>ci deploy<br/>[cloudbuild.yaml]"]
  node_deploy_sh["Deploy script<br/>ops script<br/>[deploy-cloudrun.sh]"]
  node_secrets[("Secret Manager<br/>external secret")]
end

node_run_py -->|"starts"| node_main
node_main -->|"serves"| node_health_api
node_main -->|"serves"| node_query_api
node_main -->|"serves"| node_stream_api
node_main -->|"serves"| node_cache_api
node_main -->|"orchestrates"| node_answer_flow
node_answer_flow -->|"reads"| node_pdf_source
node_answer_flow -->|"ingests"| node_loader
node_loader -->|"hands off"| node_vectorizer
node_vectorizer -->|"builds"| node_faiss_cache
node_main -->|"memoizes"| node_faiss_cache
node_answer_flow -->|"queries"| node_retriever
node_retriever -->|"searches"| node_faiss_cache
node_retriever -->|"grounds"| node_llm
node_llm -->|"streams"| node_stream_api
node_static_ui -->|"calls"| node_query_api
node_static_ui -->|"consumes"| node_stream_api
node_dockerfile -->|"packages"| node_main
node_cloudbuild -->|"builds"| node_dockerfile
node_deploy_sh -->|"invokes"| node_cloudbuild
node_deploy_sh -->|"loads"| node_secrets

click node_run_py "https://github.com/srujanrana07/policy-rag-v2/blob/main/run.py"
click node_main "https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py"
click node_loader "https://github.com/srujanrana07/policy-rag-v2/blob/main/app/document_loader.py"
click node_vectorizer "https://github.com/srujanrana07/policy-rag-v2/blob/main/app/vectorizer.py"
click node_retriever "https://github.com/srujanrana07/policy-rag-v2/blob/main/app/retriever.py"
click node_llm "https://github.com/srujanrana07/policy-rag-v2/blob/main/app/llm_client.py"
click node_static_ui "https://github.com/srujanrana07/policy-rag-v2/blob/main/static/index.html"
click node_health_api "https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py"
click node_query_api "https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py"
click node_stream_api "https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py"
click node_cache_api "https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py"
click node_dockerfile "https://github.com/srujanrana07/policy-rag-v2/tree/main/Dockerfile"
click node_cloudbuild "https://github.com/srujanrana07/policy-rag-v2/blob/main/cloudbuild.yaml"
click node_deploy_sh "https://github.com/srujanrana07/policy-rag-v2/blob/main/deploy-cloudrun.sh"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_run_py,node_main,node_loader,node_vectorizer,node_retriever,node_llm,node_faiss_cache,node_pdf_source,node_answer_flow,node_health_api,node_query_api,node_stream_api,node_cache_api toneBlue
class node_static_ui toneAmber
class node_dockerfile,node_cloudbuild,node_deploy_sh,node_secrets toneMint
```

## Components

### Application runtime

Single-process FastAPI service that ingests PDFs, retrieves evidence, and generates answers.

- [run.py](https://github.com/srujanrana07/policy-rag-v2/blob/main/run.py): dev entrypoint · Local startup entrypoint for running the service during development. · `run.py`
- [API app](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py): FastAPI orchestrator · HTTP surface, lifecycle setup, and in-memory document cache coordination. · `app/main.py`
- [Loader](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/document_loader.py): PDF ingest · Fetches remote PDFs and extracts text and tables from documents. · `app/document_loader.py`
- [Vectorizer](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/vectorizer.py): chunk+embed · Performs semantic chunking and builds embeddings for retrieval. · `app/vectorizer.py`
- [Retriever](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/retriever.py): hybrid search · Combines semantic search, lexical BM25, reranking, and query expansion. · `app/retriever.py`
- [Gemini client](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/llm_client.py): LLM wrapper · Wraps Gemini for grounded answer generation and token streaming. · `app/llm_client.py`
- **FAISS cache**: in-memory index · Process-local index and LRU cache keyed by document URL sets.
- **PDF URLs**: external input · One or more remote policy PDFs supplied by the caller.
- **Answer flow**: request pipeline · End-to-end path from question to grounded answer and optional streaming output.
- [Health](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py): endpoint · Service readiness and liveness checks. · `app/main.py`
- [Batch query](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py): endpoint · Synchronous multi-question Q&A endpoint. · `app/main.py`
- [Stream query](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py): SSE endpoint · Server-sent-events endpoint for token-by-token responses. · `app/main.py`
- [Cache ops](https://github.com/srujanrana07/policy-rag-v2/blob/main/app/main.py): endpoint · Inspection and reset endpoints for the document cache. · `app/main.py`

### User interface

Lightweight static client used to submit document URLs and questions.

- [Web client](https://github.com/srujanrana07/policy-rag-v2/blob/main/static/index.html): static frontend · Single-file browser UI for submitting URLs and questions and reading responses. · `static/index.html`

### Deployment

Containerization and Cloud Run deployment path.

- [Dockerfile](https://github.com/srujanrana07/policy-rag-v2/tree/main/Dockerfile): container build · Builds the CPU-oriented runtime image with the embedding model packaged in ONNX form. · `Dockerfile`
- [Cloud Build](https://github.com/srujanrana07/policy-rag-v2/blob/main/cloudbuild.yaml): ci deploy · Automated build-and-deploy pipeline for Cloud Run. · `cloudbuild.yaml`
- [Deploy script](https://github.com/srujanrana07/policy-rag-v2/blob/main/deploy-cloudrun.sh): ops script · Manual deployment helper for enabling APIs, building, and deploying to Cloud Run. · `deploy-cloudrun.sh`
- **Secret Manager**: external secret · External storage for the Gemini API key and other sensitive configuration.

## Connections

- run.py → API app: starts · Local entrypoint launches the FastAPI application.
- API app → Health: serves · Exposes service health checks.
- API app → Batch query: serves · Handles synchronous question answering requests.
- API app → Stream query: serves · Handles streaming SSE question requests.
- API app → Cache ops: serves · Exposes cache inspection and reset operations.
- API app → Answer flow: orchestrates · Coordinates ingestion, retrieval, and generation for each request.
- Answer flow → PDF URLs: reads · Consumes one or more remote PDF URLs.
- Answer flow → Loader: ingests · Downloads and parses the source PDFs.
- Loader → Vectorizer: hands off · Structured document content is prepared for chunking and embedding.
- Vectorizer → FAISS cache: builds · Creates embeddings and populates the searchable in-memory index.
- API app → FAISS cache: memoizes · Stores and reuses document-set indices in process-local LRU cache.
- Answer flow → Retriever: queries · Retrieval layer searches and reranks evidence for the question.
- Retriever → FAISS cache: searches · Uses semantic vectors and cached indices for candidate retrieval.
- Retriever → Gemini client: grounds · Passes retrieved context to the LLM for answer synthesis.
- Gemini client → Stream query: streams · Supports token streaming back to the SSE endpoint.
- Web client → Batch query: calls · Submits batch questions from the browser UI.
- Web client → Stream query: consumes · Reads token streams from the SSE endpoint.
- Dockerfile → API app: packages · Bundles the application and model assets into the runtime image.
- Cloud Build → Dockerfile: builds · Automated container build pipeline.
- Deploy script → Cloud Build: invokes · Manual deployment path can trigger the same build/deploy flow.
- Deploy script → Secret Manager: loads · Retrieves API credentials from Google Secret Manager.
