# lakshya-2440/hybridrag architecture

> Architecture diagram of the GitHub repository lakshya-2440/hybridrag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lakshya-2440/hybridrag
- Repository: https://github.com/lakshya-2440/hybridrag

## Overview

1. **Overall shape**
   - This repo is a two-tier RAG application: a **Next.js frontend** for document upload/chat/eval UI and a **FastAPI backend** that owns ingestion, retrieval, answer generation, and evaluation.
   - The system is designed to work in two modes: **local/free mode** using lightweight local embeddings/lexical ranking/extractive answers, and **remote model mode** using OpenRouter + Cohere.

2. **User-facing frontend**
   - The `frontend/` app is a Next.js 14 UI hosted separately from the backend.
   - Main screens/components are:
     - `DocumentUpload` and `DocumentList` for corpus management
     - `ChatInterface` for Q&A
     - `CitationCard` for source display
     - `EvalDashboard` for evaluation results
   - Frontend state/data access is abstracted through `useChat`, `useDocuments`, and `lib/api.ts`, which call the backend API.

3. **Backend API boundary**
   - The backend exposes a small REST surface under `backend/api/routes/`:
     - `documents.py` for upload/list/status operations
     - `query.py` for question answering
     - `eval.py` for dataset generation and evaluation runs
   - `backend/main.py` wires the FastAPI app, and `backend/api/schemas.py` defines request/response contracts.

4. **Ingestion pipeline**
   - Uploaded files enter `backend/ingestion/loader.py`, are normalized/chunked by `chunker.py`, and orchestrated by `pipeline.py`.
   - Ingestion produces multiple downstream artifacts: searchable chunks for retrieval, metadata for document tracking, and corpus material for evaluation dataset generation.
   - This is the boundary where raw user documents become indexed knowledge.

5. **Persistence and storage layers**
   - The architecture uses multiple storage mechanisms with different roles:
     - **ChromaDB persistent disk** for vector similarity search
     - **BM25 JSON corpus** for lexical retrieval
     - **SQLite** for metadata and evaluation logs
   - `backend/database.py` and `backend/config.py` coordinate these runtime settings and paths.

6. **Hybrid retrieval path**
   - Retrieval is explicitly hybrid, combining dense and lexical strategies.
   - `vector_store.py` handles embedding-based lookup, `bm25_store.py` handles term-based retrieval, and `hybrid.py` fuses the candidate sets.
   - `reranker.py` then refines the retrieved passages, using Cohere when remote mode is enabled or local lexical logic when not.

7. **Answer generation and citation enforcement**
   - `backend/generation/chain.py` is the answer assembly layer: it takes the top evidence, builds prompts, and returns grounded responses.
   - Prompt files in `backend/generation/prompts/` control system behavior, rewrite behavior, and citation enforcement.
   - `citations.py` formats and validates source citations so answers are tied to retrieved chunks instead of free-form generation.

8. **External model dependencies**
   - In remote mode, the backend calls **OpenRouter** for LLM inference and embeddings, and **Cohere** for reranking.
   - The README makes clear these are optional; the default local mode avoids paid API use and is intended to still work with the same architecture.

9. **Evaluation subsystem**
   - The `backend/evaluation/` package is a first-class subsystem, not an afterthought.
   - `dataset_generator.py` builds a golden QA set from ingested documents using RAGAS TestsetGenerator.
   - `eval_runner.py` executes the evaluation, and `metrics.py` computes quality scores such as faithfulness and answer relevancy.
   - Results are surfaced in the frontend eval page and persisted/logged via the backend.

10. **Data flow end-to-end**
    - The core runtime flow is: **upload document → ingest/chunk/index → retrieve relevant evidence → rerank → generate cited answer → display in chat**.
    - The eval flow is: **documents → synthetic QA dataset generation → automated metric run → dashboard/CI gating**.
    - This means the same corpus powers both user Q&A and quality measurement.

11. **Deployment boundaries**
    - The system is deployed as two independently hosted services:
      - **Frontend on Vercel**
      - **Backend on Render**
    - `frontend/vercel.json` and the root/Vercel config support frontend deployment, while the backend is expected to run as a FastAPI service with persistent storage available to Chroma/SQLite/BM25.

12. **CI and quality gates**
    - GitHub Actions in `.github/workflows/ci.yml` run automated checks on push.
    - The README indicates CI includes evaluation gating: builds fail if **faithfulness < 0.80** or **answer relevancy < 0.75**.
    - This makes evaluation part of the delivery pipeline, not just a manual benchmark.

13. **Testing strategy**
    - `backend/tests/` covers API, ingestion, retrieval, and generation as separate concerns.
    - This mirrors the architecture: each major subsystem has isolated tests, which helps preserve the contract between ingestion, retrieval, generation, and evaluation.

14. **How to think about the architecture diagram**
    - Draw the backend as four internal layers: **API**, **ingestion/indexing**, **retrieval**, and **generation/evaluation**.
    - Draw the storage/services beneath it: **ChromaDB**, **BM25 corpus**, **SQLite**, plus **OpenRouter** and **Cohere** as optional external dependencies.
    - Keep the frontend as a thin client that talks only to the backend API and renders chat, documents, citations, and eval results.

Main entry points: [`backend/ingestion/pipeline.py`](https://github.com/lakshya-2440/hybridrag/blob/main/backend/ingestion/pipeline.py) (Ingest Pipeline), [`frontend/src/app`](https://github.com/lakshya-2440/hybridrag/tree/main/frontend/src/app) (Next.js App), [`frontend/src/lib/api.ts`](https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/lib/api.ts) (API Client), [`backend/api/routes/eval.py`](https://github.com/lakshya-2440/hybridrag/blob/main/backend/api/routes/eval.py) (Eval API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend"]
  node_frontend_app["Next.js App<br/>UI shell"]
  node_chat_ui["Chat UI<br/>[ChatInterface.tsx]"]
  node_doc_upload_ui["Upload Docs<br/>[DocumentUpload.tsx]"]
  node_doc_list_ui["Document List<br/>[DocumentList.tsx]"]
  node_citation_ui["Citations<br/>[CitationCard.tsx]"]
  node_eval_ui["Eval Dashboard<br/>page<br/>[page.tsx]"]
  node_chat_hook["Chat Hook<br/>state hook<br/>[useChat.ts]"]
  node_docs_hook["Docs Hook<br/>state hook<br/>[useDocuments.ts]"]
  node_api_client["API Client<br/>frontend adapter<br/>[api.ts]"]
end

subgraph group_api["Backend API"]
  node_main_app["FastAPI App<br/>server entry<br/>[main.py]"]
  node_api_schemas["API Schemas<br/>contracts<br/>[schemas.py]"]
  node_documents_route["Documents API<br/>route<br/>[documents.py]"]
  node_query_route["Query API<br/>route<br/>[query.py]"]
  node_eval_route["Eval API<br/>route<br/>[eval.py]"]
end

subgraph group_ingest["Ingestion"]
  node_ingest_pipeline["Ingest Pipeline<br/>orchestrator<br/>[pipeline.py]"]
  node_loader["Loader<br/>[loader.py]"]
  node_chunker["Chunker<br/>text splitter<br/>[chunker.py]"]
end

subgraph group_retrieve["Retrieval"]
  node_vector_store[("Vector Store<br/>dense retrieval<br/>[vector_store.py]")]
  node_bm25_store[("BM25 Store<br/>lexical retrieval<br/>[bm25_store.py]")]
  node_hybrid_retrieval["Hybrid Fusion<br/>retrieval combiner<br/>[hybrid.py]"]
  node_reranker["Reranker<br/>ranking stage<br/>[reranker.py]"]
end

subgraph group_generate["Generation"]
  node_answer_chain["Answer Chain<br/>generation pipeline<br/>[chain.py]"]
  node_citations["Citation Logic<br/>citation validator<br/>[citations.py]"]
  node_prompt_files["Prompts<br/>prompt assets"]
end

subgraph group_eval["Evaluation"]
  node_dataset_gen["Dataset Gen<br/>evaluation builder"]
  node_eval_runner["Eval Runner<br/>evaluation runner<br/>[eval_runner.py]"]
  node_metrics["Metrics<br/>scoring<br/>[metrics.py]"]
end

subgraph group_storage["Storage"]
  node_config_db["Config + DB<br/>runtime config<br/>[config.py]"]
  node_sqlite_db[("SQLite<br/>metadata store<br/>[database.py]")]
  node_external_models(("OpenRouter<br/>external LLM/embeddings"))
  node_external_cohere(("Cohere<br/>external reranker"))
end

node_frontend_app -->|"renders"| node_chat_ui
node_frontend_app -->|"renders"| node_doc_upload_ui
node_frontend_app -->|"renders"| node_doc_list_ui
node_frontend_app -->|"renders"| node_citation_ui
node_frontend_app -->|"renders"| node_eval_ui
node_chat_ui -->|"uses"| node_chat_hook
node_doc_upload_ui -->|"uses"| node_docs_hook
node_doc_list_ui -->|"uses"| node_docs_hook
node_chat_hook -->|"calls"| node_api_client
node_docs_hook -->|"calls"| node_api_client
node_api_client -->|"requests"| node_documents_route
node_api_client -->|"requests"| node_query_route
node_api_client -->|"requests"| node_eval_route
node_main_app -->|"mounts"| node_documents_route
node_main_app -->|"mounts"| node_query_route
node_main_app -->|"mounts"| node_eval_route
node_documents_route -->|"validates"| node_api_schemas
node_query_route -->|"validates"| node_api_schemas
node_eval_route -->|"validates"| node_api_schemas
node_documents_route -->|"triggers"| node_ingest_pipeline
node_ingest_pipeline -->|"loads"| node_loader
node_ingest_pipeline -->|"chunks"| node_chunker
node_ingest_pipeline -->|"indexes"| node_vector_store
node_ingest_pipeline -->|"indexes"| node_bm25_store
node_query_route -->|"retrieves"| node_hybrid_retrieval
node_hybrid_retrieval -->|"queries"| node_vector_store
node_hybrid_retrieval -->|"queries"| node_bm25_store
node_hybrid_retrieval -->|"passes"| node_reranker
node_reranker -.->|"optional"| node_external_cohere
node_reranker -->|"supplies"| node_answer_chain
node_answer_chain -->|"uses"| node_prompt_files
node_answer_chain -->|"enforces"| node_citations
node_answer_chain -.->|"optional"| node_external_models
node_eval_route -->|"builds"| node_dataset_gen
node_eval_route -->|"runs"| node_eval_runner
node_eval_runner -->|"scores"| node_metrics
node_dataset_gen -->|"reuses"| node_hybrid_retrieval
node_config_db -->|"configures"| node_sqlite_db
node_ingest_pipeline -->|"reads"| node_config_db
node_eval_runner -->|"logs"| node_sqlite_db

click node_frontend_app "https://github.com/lakshya-2440/hybridrag/tree/main/frontend/src/app"
click node_chat_ui "https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/components/ChatInterface.tsx"
click node_doc_upload_ui "https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/components/DocumentUpload.tsx"
click node_doc_list_ui "https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/components/DocumentList.tsx"
click node_citation_ui "https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/components/CitationCard.tsx"
click node_eval_ui "https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/app/eval/page.tsx"
click node_chat_hook "https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/hooks/useChat.ts"
click node_docs_hook "https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/hooks/useDocuments.ts"
click node_api_client "https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/lib/api.ts"
click node_main_app "https://github.com/lakshya-2440/hybridrag/blob/main/backend/main.py"
click node_api_schemas "https://github.com/lakshya-2440/hybridrag/blob/main/backend/api/schemas.py"
click node_documents_route "https://github.com/lakshya-2440/hybridrag/blob/main/backend/api/routes/documents.py"
click node_query_route "https://github.com/lakshya-2440/hybridrag/blob/main/backend/api/routes/query.py"
click node_eval_route "https://github.com/lakshya-2440/hybridrag/blob/main/backend/api/routes/eval.py"
click node_ingest_pipeline "https://github.com/lakshya-2440/hybridrag/blob/main/backend/ingestion/pipeline.py"
click node_loader "https://github.com/lakshya-2440/hybridrag/blob/main/backend/ingestion/loader.py"
click node_chunker "https://github.com/lakshya-2440/hybridrag/blob/main/backend/ingestion/chunker.py"
click node_vector_store "https://github.com/lakshya-2440/hybridrag/blob/main/backend/retrieval/vector_store.py"
click node_bm25_store "https://github.com/lakshya-2440/hybridrag/blob/main/backend/retrieval/bm25_store.py"
click node_hybrid_retrieval "https://github.com/lakshya-2440/hybridrag/blob/main/backend/retrieval/hybrid.py"
click node_reranker "https://github.com/lakshya-2440/hybridrag/blob/main/backend/retrieval/reranker.py"
click node_answer_chain "https://github.com/lakshya-2440/hybridrag/blob/main/backend/generation/chain.py"
click node_citations "https://github.com/lakshya-2440/hybridrag/blob/main/backend/generation/citations.py"
click node_prompt_files "https://github.com/lakshya-2440/hybridrag/tree/main/backend/generation/prompts"
click node_dataset_gen "https://github.com/lakshya-2440/hybridrag/blob/main/backend/evaluation/dataset_generator.py"
click node_eval_runner "https://github.com/lakshya-2440/hybridrag/blob/main/backend/evaluation/eval_runner.py"
click node_metrics "https://github.com/lakshya-2440/hybridrag/blob/main/backend/evaluation/metrics.py"
click node_config_db "https://github.com/lakshya-2440/hybridrag/blob/main/backend/config.py"
click node_sqlite_db "https://github.com/lakshya-2440/hybridrag/blob/main/backend/database.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend_app,node_chat_ui,node_doc_upload_ui,node_doc_list_ui,node_citation_ui,node_eval_ui,node_chat_hook,node_docs_hook,node_api_client,node_config_db,node_sqlite_db,node_external_models,node_external_cohere toneBlue
class node_main_app,node_api_schemas,node_documents_route,node_query_route,node_eval_route toneAmber
class node_ingest_pipeline,node_loader,node_chunker toneMint
class node_vector_store,node_bm25_store,node_hybrid_retrieval,node_reranker toneRose
class node_answer_chain,node_citations,node_prompt_files toneIndigo
class node_dataset_gen,node_eval_runner,node_metrics toneTeal
```

## Components

### Frontend

Next.js UI for upload, chat, citations, and evaluation

- [Next.js App](https://github.com/lakshya-2440/hybridrag/tree/main/frontend/src/app): UI shell · Frontend entrypoint and route structure · `frontend/src/app`
- [Chat UI](https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/components/ChatInterface.tsx): Main Q&A interface · `frontend/src/components/ChatInterface.tsx`
- [Upload Docs](https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/components/DocumentUpload.tsx): Corpus upload control · `frontend/src/components/DocumentUpload.tsx`
- [Document List](https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/components/DocumentList.tsx): Uploaded document browser · `frontend/src/components/DocumentList.tsx`
- [Citations](https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/components/CitationCard.tsx): Source display for answers · `frontend/src/components/CitationCard.tsx`
- [Eval Dashboard](https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/app/eval/page.tsx): page · Evaluation results view · `frontend/src/app/eval/page.tsx`
- [Chat Hook](https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/hooks/useChat.ts): state hook · Chat state and API orchestration · `frontend/src/hooks/useChat.ts`
- [Docs Hook](https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/hooks/useDocuments.ts): state hook · Document state and API orchestration · `frontend/src/hooks/useDocuments.ts`
- [API Client](https://github.com/lakshya-2440/hybridrag/blob/main/frontend/src/lib/api.ts): frontend adapter · Browser-to-backend request wrapper · `frontend/src/lib/api.ts`

### Backend API

FastAPI boundary and request contracts

- [FastAPI App](https://github.com/lakshya-2440/hybridrag/blob/main/backend/main.py): server entry · Backend application wiring · `backend/main.py`
- [API Schemas](https://github.com/lakshya-2440/hybridrag/blob/main/backend/api/schemas.py): contracts · Request and response models · `backend/api/schemas.py`
- [Documents API](https://github.com/lakshya-2440/hybridrag/blob/main/backend/api/routes/documents.py): route · Upload, list, and status endpoints · `backend/api/routes/documents.py`
- [Query API](https://github.com/lakshya-2440/hybridrag/blob/main/backend/api/routes/query.py): route · Question answering endpoint · `backend/api/routes/query.py`
- [Eval API](https://github.com/lakshya-2440/hybridrag/blob/main/backend/api/routes/eval.py): route · Dataset and evaluation endpoints · `backend/api/routes/eval.py`

### Ingestion

Document loading, chunking, and indexing pipeline

- [Ingest Pipeline](https://github.com/lakshya-2440/hybridrag/blob/main/backend/ingestion/pipeline.py): orchestrator · Coordinates document processing · `backend/ingestion/pipeline.py`
- [Loader](https://github.com/lakshya-2440/hybridrag/blob/main/backend/ingestion/loader.py): document loader · Reads and normalizes uploaded files · `backend/ingestion/loader.py`
- [Chunker](https://github.com/lakshya-2440/hybridrag/blob/main/backend/ingestion/chunker.py): text splitter · Segments documents into chunks · `backend/ingestion/chunker.py`

### Retrieval

Hybrid evidence search and reranking

- [Vector Store](https://github.com/lakshya-2440/hybridrag/blob/main/backend/retrieval/vector_store.py): dense retrieval · Embedding-backed similarity search · `backend/retrieval/vector_store.py`
- [BM25 Store](https://github.com/lakshya-2440/hybridrag/blob/main/backend/retrieval/bm25_store.py): lexical retrieval · Term-based corpus lookup · `backend/retrieval/bm25_store.py`
- [Hybrid Fusion](https://github.com/lakshya-2440/hybridrag/blob/main/backend/retrieval/hybrid.py): retrieval combiner · Merges dense and lexical candidates · `backend/retrieval/hybrid.py`
- [Reranker](https://github.com/lakshya-2440/hybridrag/blob/main/backend/retrieval/reranker.py): ranking stage · Refines evidence ordering · `backend/retrieval/reranker.py`

### Generation

Answer assembly, prompting, and citation control

- [Answer Chain](https://github.com/lakshya-2440/hybridrag/blob/main/backend/generation/chain.py): generation pipeline · Builds grounded responses from evidence · `backend/generation/chain.py`
- [Citation Logic](https://github.com/lakshya-2440/hybridrag/blob/main/backend/generation/citations.py): citation validator · Formats and enforces source citations · `backend/generation/citations.py`
- [Prompts](https://github.com/lakshya-2440/hybridrag/tree/main/backend/generation/prompts): prompt assets · System, rewrite, and citation prompts · `backend/generation/prompts`

### Evaluation

Dataset generation, scoring, and run orchestration

- [Dataset Gen](https://github.com/lakshya-2440/hybridrag/blob/main/backend/evaluation/dataset_generator.py): evaluation builder · Creates golden QA sets from corpus · `backend/evaluation/dataset_generator.py`
- [Eval Runner](https://github.com/lakshya-2440/hybridrag/blob/main/backend/evaluation/eval_runner.py): evaluation runner · Executes automated evaluation runs · `backend/evaluation/eval_runner.py`
- [Metrics](https://github.com/lakshya-2440/hybridrag/blob/main/backend/evaluation/metrics.py): scoring · Computes faithfulness and relevancy scores · `backend/evaluation/metrics.py`

### Storage

Persistent stores and external model services

- [Config + DB](https://github.com/lakshya-2440/hybridrag/blob/main/backend/config.py): runtime config · Paths, persistence, and service settings · `backend/config.py`
- [SQLite](https://github.com/lakshya-2440/hybridrag/blob/main/backend/database.py): metadata store · Metadata and evaluation logs · `backend/database.py`
- **OpenRouter**: external LLM/embeddings · Optional remote inference and embeddings
- **Cohere**: external reranker · Optional remote reranking service

## Connections

- Next.js App → Chat UI: renders · App composes chat experience
- Next.js App → Upload Docs: renders · App composes upload controls
- Next.js App → Document List: renders · App composes document browser
- Next.js App → Citations: renders · App displays cited sources
- Next.js App → Eval Dashboard: renders · App displays evaluation dashboard
- Chat UI → Chat Hook: uses · Chat UI delegates state and actions
- Upload Docs → Docs Hook: uses · Upload UI updates document state
- Document List → Docs Hook: uses · Document list reads backend state
- Chat Hook → API Client: calls · Chat actions invoke backend APIs
- Docs Hook → API Client: calls · Document actions invoke backend APIs
- API Client → Documents API: requests · Frontend document operations
- API Client → Query API: requests · Frontend question answering requests
- API Client → Eval API: requests · Frontend evaluation requests
- FastAPI App → Documents API: mounts · Registers document endpoints
- FastAPI App → Query API: mounts · Registers query endpoints
- FastAPI App → Eval API: mounts · Registers evaluation endpoints
- Documents API → API Schemas: validates · Uses contracts for document requests
- Query API → API Schemas: validates · Uses contracts for query requests
- Eval API → API Schemas: validates · Uses contracts for eval requests
- Documents API → Ingest Pipeline: triggers · Upload flow starts ingestion
- Ingest Pipeline → Loader: loads · Reads raw documents
- Ingest Pipeline → Chunker: chunks · Normalizes content into chunks
- Ingest Pipeline → Vector Store: indexes · Writes dense retrieval index
- Ingest Pipeline → BM25 Store: indexes · Writes lexical retrieval corpus
- Query API → Hybrid Fusion: retrieves · Question answering starts evidence search
- Hybrid Fusion → Vector Store: queries · Fetches dense candidates
- Hybrid Fusion → BM25 Store: queries · Fetches lexical candidates
- Hybrid Fusion → Reranker: passes · Merged candidates are reranked
- Reranker → Cohere: optional · Uses Cohere in remote mode
- Reranker → Answer Chain: supplies · Top evidence feeds answer generation
- Answer Chain → Prompts: uses · Prompt templates guide generation
- Answer Chain → Citation Logic: enforces · Citations are formatted and checked
- Answer Chain → OpenRouter: optional · Uses remote LLMs and embeddings when enabled
- Eval API → Dataset Gen: builds · Generates synthetic QA datasets
- Eval API → Eval Runner: runs · Starts evaluation execution
- Eval Runner → Metrics: scores · Computes quality metrics
- Dataset Gen → Hybrid Fusion: reuses · Evaluation set is grounded in corpus access
- Config + DB → SQLite: configures · Defines persistence and runtime paths
- Ingest Pipeline → Config + DB: reads · Uses runtime storage settings
- Eval Runner → SQLite: logs · Persists evaluation outputs