How srujanrana07/policy-rag-v2 works
1. What this repository is
- A single-service RAG application for answering questions over insurance policy PDFs.
- The system is built around a FastAPI backend that orchestrates PDF ingestion, retrieval, and Gemini-based answer generation.
2. Primary runtime boundary
- The core application lives in app/ and is deployed as one container.
- app/main.py is the orchestration layer: it exposes HTTP endpoints, manages lifecycle setup, and holds the in-memory document cache.
- run.py is the local dev entrypoint; Docker/Cloud Run is the production runtime.
3. Document ingestion pipeline
- document_loader.py handles remote PDF retrieval and parsing.
- Text extraction uses PyMuPDF, with table extraction via Camelot lattice mode and pdfplumber as fallback.
- This stage turns one or more PDF URLs into structured text/table content ready for chunking.
4. Chunking and embedding
- vectorizer.py is responsible for semantic chunking and building embeddings.
- The embedding model is Sentence-Transformers all-MiniLM-L6-v2, exported to ONNX for CPU inference.
- Output from this stage becomes the basis for similarity search and FAISS indexing.
5. Search index and cache
- Each unique set of document URLs maps to an in-memory FAISS index.
- main.py keeps an LRU cache of these document sets so repeated questions over the same PDFs skip download, extraction, and embedding.
- The cache is process-local and intentionally ephemeral, matching Cloud Run’s stateless container model.
6. Retrieval layer
- retriever.py implements the document search stack.
- The README describes hybrid retrieval: semantic search from FAISS, lexical BM25, and a custom reranker.
- Query expansion is part of this layer too, using abbreviation/synonym handling for insurance terms like PED, ICU, SI, and NCB.
7. LLM answer generation
- llm_client.py wraps Google Gemini 2.5 Flash.
- It supports both synchronous question answering and token-streaming output for the SSE endpoint.
- Retrieval results are passed to Gemini as grounded context for final answer synthesis.
8. API surface
- The backend exposes a small set of endpoints: health, synchronous multi-question query, streaming query, cache inspection, and cache reset.
- POST /query is the batch Q&A path.
- GET /stream-query is the interactive path, returning server-sent events token by token.
9. UI boundary
- static/index.html is a single-file frontend with no separate build system.
- It is a lightweight client for submitting document URLs/questions and consuming either JSON responses or SSE streams.
- The README notes that users point the UI at the deployed Cloud Run service URL.
10. Cold-start and model packaging
- The Docker image bakes in the embedding model in ONNX form.
- This avoids PyTorch at runtime and prevents network downloads for model weights during cold start.
- The architecture is optimized for CPU-only inference on Cloud Run.
11. Deployment and CI/CD
- cloudbuild.yaml supports automated build-and-deploy from source.
- deploy-cloudrun.sh is the manual path that enables APIs, builds the container, and deploys to Cloud Run.
- Secret handling is externalized through Google Secret Manager for the Gemini API key.
12. Operational model
- The service is designed to scale horizontally through Cloud Run instances, not multiple workers in one process.
- The README explicitly notes workers 1 because the FAISS index and LRU cache live in memory inside a single process.
- min-instances=0 keeps the service cost-efficient when idle.
13. End-to-end data flow
- Input: one or more PDF URLs plus a question or set of questions.
- Processing: download → extract text/tables → chunk → embed → index/cache.
- Retrieval: embed question → hybrid search/rerank → provide evidence context.
- Generation: Gemini produces the answer, either as a JSON response or streamed SSE tokens.
14. Key architectural constraint
- The central tradeoff is speed versus persistence: caching and FAISS live in memory for fast repeated access, but nothing survives container restarts.
- That makes the service simple and Cloud Run-friendly, but also means every fresh instance rebuilds document indices on demand.
Main entry points: main.py API app · retriever.py Retriever · index.html Web client · deploy-cloudrun.sh Deploy script
Components 18
Application runtime
Single-process FastAPI service that ingests PDFs, retrieves evidence, and generates answers.
- run.py · Local startup entrypoint for running the service during development.
run.py - API app · HTTP surface, lifecycle setup, and in-memory document cache coordination.
app/main.py - Loader · Fetches remote PDFs and extracts text and tables from documents.
app/document_loader.py - Vectorizer · Performs semantic chunking and builds embeddings for retrieval.
app/vectorizer.py - Retriever · Combines semantic search, lexical BM25, reranking, and query expansion.
app/retriever.py - Gemini client · Wraps Gemini for grounded answer generation and token streaming.
app/llm_client.py - FAISS cache · Process-local index and LRU cache keyed by document URL sets.
- PDF URLs · One or more remote policy PDFs supplied by the caller.
- Answer flow · End-to-end path from question to grounded answer and optional streaming output.
- Health · Service readiness and liveness checks.
app/main.py - Batch query · Synchronous multi-question Q&A endpoint.
app/main.py - Stream query · Server-sent-events endpoint for token-by-token responses.
app/main.py - Cache ops · Inspection and reset endpoints for the document cache.
app/main.py
User interface
Lightweight static client used to submit document URLs and questions.
- Web client · Single-file browser UI for submitting URLs and questions and reading responses.
static/index.html
Deployment
Containerization and Cloud Run deployment path.
- Dockerfile · Builds the CPU-oriented runtime image with the embedding model packaged in ONNX form.
Dockerfile - Cloud Build · Automated build-and-deploy pipeline for Cloud Run.
cloudbuild.yaml - Deploy script · Manual deployment helper for enabling APIs, building, and deploying to Cloud Run.
deploy-cloudrun.sh - Secret Manager · External storage for the Gemini API key and other sensitive configuration.
Connections 21
- run.py → API app: starts · Local entrypoint launches the FastAPI application.
- API app → Health: serves · Exposes service health checks.
- API app → Batch query: serves · Handles synchronous question answering requests.
- API app → Stream query: serves · Handles streaming SSE question requests.
- API app → Cache ops: serves · Exposes cache inspection and reset operations.
- API app → Answer flow: orchestrates · Coordinates ingestion, retrieval, and generation for each request.
- Answer flow → PDF URLs: reads · Consumes one or more remote PDF URLs.
- Answer flow → Loader: ingests · Downloads and parses the source PDFs.
- Loader → Vectorizer: hands off · Structured document content is prepared for chunking and embedding.
- Vectorizer → FAISS cache: builds · Creates embeddings and populates the searchable in-memory index.
- API app → FAISS cache: memoizes · Stores and reuses document-set indices in process-local LRU cache.
- Answer flow → Retriever: queries · Retrieval layer searches and reranks evidence for the question.
- Retriever → FAISS cache: searches · Uses semantic vectors and cached indices for candidate retrieval.
- Retriever → Gemini client: grounds · Passes retrieved context to the LLM for answer synthesis.
- Gemini client → Stream query: streams · Supports token streaming back to the SSE endpoint.
- Web client → Batch query: calls · Submits batch questions from the browser UI.
- Web client → Stream query: consumes · Reads token streams from the SSE endpoint.
- Dockerfile → API app: packages · Bundles the application and model assets into the runtime image.
- Cloud Build → Dockerfile: builds · Automated container build pipeline.
- Deploy script → Cloud Build: invokes · Manual deployment path can trigger the same build/deploy flow.
- Deploy script → Secret Manager: loads · Retrieves API credentials from Google Secret Manager.