How lakshya-2440/hybridrag works
1. Overall shape
- This repo is a two-tier RAG application: a Next.js frontend for document upload/chat/eval UI and a FastAPI backend that owns ingestion, retrieval, answer generation, and evaluation.
- The system is designed to work in two modes: local/free mode using lightweight local embeddings/lexical ranking/extractive answers, and remote model mode using OpenRouter + Cohere.
2. User-facing frontend
- The frontend/ app is a Next.js 14 UI hosted separately from the backend.
- Main screens/components are:
- DocumentUpload and DocumentList for corpus management
- ChatInterface for Q&A
- CitationCard for source display
- EvalDashboard for evaluation results
- Frontend state/data access is abstracted through useChat, useDocuments, and lib/api.ts, which call the backend API.
3. Backend API boundary
- The backend exposes a small REST surface under backend/api/routes/:
- documents.py for upload/list/status operations
- query.py for question answering
- eval.py for dataset generation and evaluation runs
- backend/main.py wires the FastAPI app, and backend/api/schemas.py defines request/response contracts.
4. Ingestion pipeline
- Uploaded files enter backend/ingestion/loader.py, are normalized/chunked by chunker.py, and orchestrated by pipeline.py.
- Ingestion produces multiple downstream artifacts: searchable chunks for retrieval, metadata for document tracking, and corpus material for evaluation dataset generation.
- This is the boundary where raw user documents become indexed knowledge.
5. Persistence and storage layers
- The architecture uses multiple storage mechanisms with different roles:
- ChromaDB persistent disk for vector similarity search
- BM25 JSON corpus for lexical retrieval
- SQLite for metadata and evaluation logs
- backend/database.py and backend/config.py coordinate these runtime settings and paths.
6. Hybrid retrieval path
- Retrieval is explicitly hybrid, combining dense and lexical strategies.
- vector_store.py handles embedding-based lookup, bm25_store.py handles term-based retrieval, and hybrid.py fuses the candidate sets.
- reranker.py then refines the retrieved passages, using Cohere when remote mode is enabled or local lexical logic when not.
7. Answer generation and citation enforcement
- backend/generation/chain.py is the answer assembly layer: it takes the top evidence, builds prompts, and returns grounded responses.
- Prompt files in backend/generation/prompts/ control system behavior, rewrite behavior, and citation enforcement.
- citations.py formats and validates source citations so answers are tied to retrieved chunks instead of free-form generation.
8. External model dependencies
- In remote mode, the backend calls OpenRouter for LLM inference and embeddings, and Cohere for reranking.
- The README makes clear these are optional; the default local mode avoids paid API use and is intended to still work with the same architecture.
9. Evaluation subsystem
- The backend/evaluation/ package is a first-class subsystem, not an afterthought.
- dataset_generator.py builds a golden QA set from ingested documents using RAGAS TestsetGenerator.
- eval_runner.py executes the evaluation, and metrics.py computes quality scores such as faithfulness and answer relevancy.
- Results are surfaced in the frontend eval page and persisted/logged via the backend.
10. Data flow end-to-end
- The core runtime flow is: upload document → ingest/chunk/index → retrieve relevant evidence → rerank → generate cited answer → display in chat.
- The eval flow is: documents → synthetic QA dataset generation → automated metric run → dashboard/CI gating.
- This means the same corpus powers both user Q&A and quality measurement.
11. Deployment boundaries
- The system is deployed as two independently hosted services:
- Frontend on Vercel
- Backend on Render
- frontend/vercel.json and the root/Vercel config support frontend deployment, while the backend is expected to run as a FastAPI service with persistent storage available to Chroma/SQLite/BM25.
12. CI and quality gates
- GitHub Actions in .github/workflows/ci.yml run automated checks on push.
- The README indicates CI includes evaluation gating: builds fail if faithfulness < 0.80 or answer relevancy < 0.75.
- This makes evaluation part of the delivery pipeline, not just a manual benchmark.
13. Testing strategy
- backend/tests/ covers API, ingestion, retrieval, and generation as separate concerns.
- This mirrors the architecture: each major subsystem has isolated tests, which helps preserve the contract between ingestion, retrieval, generation, and evaluation.
14. How to think about the architecture diagram
- Draw the backend as four internal layers: API, ingestion/indexing, retrieval, and generation/evaluation.
- Draw the storage/services beneath it: ChromaDB, BM25 corpus, SQLite, plus OpenRouter and Cohere as optional external dependencies.
- Keep the frontend as a thin client that talks only to the backend API and renders chat, documents, citations, and eval results.
Main entry points: pipeline.py Ingest Pipeline · app Next.js App · api.ts API Client · eval.py Eval API
Components 31
Frontend
Next.js UI for upload, chat, citations, and evaluation
- Next.js App · Frontend entrypoint and route structure
frontend/src/app - Chat UI · Main Q&A interface
frontend/src/components/ChatInterface.tsx - Upload Docs · Corpus upload control
frontend/src/components/DocumentUpload.tsx - Document List · Uploaded document browser
frontend/src/components/DocumentList.tsx - Citations · Source display for answers
frontend/src/components/CitationCard.tsx - Eval Dashboard · Evaluation results view
frontend/src/app/eval/page.tsx - Chat Hook · Chat state and API orchestration
frontend/src/hooks/useChat.ts - Docs Hook · Document state and API orchestration
frontend/src/hooks/useDocuments.ts - API Client · Browser-to-backend request wrapper
frontend/src/lib/api.ts
Backend API
FastAPI boundary and request contracts
- FastAPI App · Backend application wiring
backend/main.py - API Schemas · Request and response models
backend/api/schemas.py - Documents API · Upload, list, and status endpoints
backend/api/routes/documents.py - Query API · Question answering endpoint
backend/api/routes/query.py - Eval API · Dataset and evaluation endpoints
backend/api/routes/eval.py
Ingestion
Document loading, chunking, and indexing pipeline
- Ingest Pipeline · Coordinates document processing
backend/ingestion/pipeline.py - Loader · Reads and normalizes uploaded files
backend/ingestion/loader.py - Chunker · Segments documents into chunks
backend/ingestion/chunker.py
Retrieval
Hybrid evidence search and reranking
- Vector Store · Embedding-backed similarity search
backend/retrieval/vector_store.py - BM25 Store · Term-based corpus lookup
backend/retrieval/bm25_store.py - Hybrid Fusion · Merges dense and lexical candidates
backend/retrieval/hybrid.py - Reranker · Refines evidence ordering
backend/retrieval/reranker.py
Generation
Answer assembly, prompting, and citation control
- Answer Chain · Builds grounded responses from evidence
backend/generation/chain.py - Citation Logic · Formats and enforces source citations
backend/generation/citations.py - Prompts · System, rewrite, and citation prompts
backend/generation/prompts
Evaluation
Dataset generation, scoring, and run orchestration
- Dataset Gen · Creates golden QA sets from corpus
backend/evaluation/dataset_generator.py - Eval Runner · Executes automated evaluation runs
backend/evaluation/eval_runner.py - Metrics · Computes faithfulness and relevancy scores
backend/evaluation/metrics.py
Storage
Persistent stores and external model services
- Config + DB · Paths, persistence, and service settings
backend/config.py - SQLite · Metadata and evaluation logs
backend/database.py - OpenRouter · Optional remote inference and embeddings
- Cohere · Optional remote reranking service
Connections 40
- Next.js App → Chat UI: renders · App composes chat experience
- Next.js App → Upload Docs: renders · App composes upload controls
- Next.js App → Document List: renders · App composes document browser
- Next.js App → Citations: renders · App displays cited sources
- Next.js App → Eval Dashboard: renders · App displays evaluation dashboard
- Chat UI → Chat Hook: uses · Chat UI delegates state and actions
- Upload Docs → Docs Hook: uses · Upload UI updates document state
- Document List → Docs Hook: uses · Document list reads backend state
- Chat Hook → API Client: calls · Chat actions invoke backend APIs
- Docs Hook → API Client: calls · Document actions invoke backend APIs
- API Client → Documents API: requests · Frontend document operations
- API Client → Query API: requests · Frontend question answering requests
- API Client → Eval API: requests · Frontend evaluation requests
- FastAPI App → Documents API: mounts · Registers document endpoints
- FastAPI App → Query API: mounts · Registers query endpoints
- FastAPI App → Eval API: mounts · Registers evaluation endpoints
- Documents API → API Schemas: validates · Uses contracts for document requests
- Query API → API Schemas: validates · Uses contracts for query requests
- Eval API → API Schemas: validates · Uses contracts for eval requests
- Documents API → Ingest Pipeline: triggers · Upload flow starts ingestion
- Ingest Pipeline → Loader: loads · Reads raw documents
- Ingest Pipeline → Chunker: chunks · Normalizes content into chunks
- Ingest Pipeline → Vector Store: indexes · Writes dense retrieval index
- Ingest Pipeline → BM25 Store: indexes · Writes lexical retrieval corpus
- Query API → Hybrid Fusion: retrieves · Question answering starts evidence search
- Hybrid Fusion → Vector Store: queries · Fetches dense candidates
- Hybrid Fusion → BM25 Store: queries · Fetches lexical candidates
- Hybrid Fusion → Reranker: passes · Merged candidates are reranked
- Reranker → Cohere: optional · Uses Cohere in remote mode
- Reranker → Answer Chain: supplies · Top evidence feeds answer generation
- Answer Chain → Prompts: uses · Prompt templates guide generation
- Answer Chain → Citation Logic: enforces · Citations are formatted and checked
- Answer Chain → OpenRouter: optional · Uses remote LLMs and embeddings when enabled
- Eval API → Dataset Gen: builds · Generates synthetic QA datasets
- Eval API → Eval Runner: runs · Starts evaluation execution
- Eval Runner → Metrics: scores · Computes quality metrics
- Dataset Gen → Hybrid Fusion: reuses · Evaluation set is grounded in corpus access
- Config + DB → SQLite: configures · Defines persistence and runtime paths
- Ingest Pipeline → Config + DB: reads · Uses runtime storage settings
- Eval Runner → SQLite: logs · Persists evaluation outputs