# thegreatpratyush/rag architecture

> Architecture diagram of the GitHub repository thegreatpratyush/rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thegreatpratyush/rag
- Repository: https://github.com/thegreatpratyush/rag

## Overview

Local RAG is a document question-answering product: users upload files, which are parsed, cleaned, chunked, embedded, and stored in dense and lexical indexes; questions are searched through hybrid retrieval, reranked, answered by a local LLM, and returned with page-aware citations or a refusal. The graph captures the documented product capabilities and source-supported orchestration. Major retrieval internals not individually excerpted are included where the README and source index establish them; those links are documented rather than inferred from module co-location.

Main entry points: [`src/pipeline/rag_pipeline.py`](https://github.com/thegreatpratyush/rag/blob/main/src/pipeline/rag_pipeline.py) (RAG pipeline), [`src/api/routes.py`](https://github.com/thegreatpratyush/rag/blob/main/src/api/routes.py) (Upload and ask routes), [`src/retrieval/vector_store.py`](https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/vector_store.py) (FAISS vector store), [`src/retrieval/index_manager.py`](https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/index_manager.py) (FAISS index manager)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["User interface"]
  node_web["RAG interface<br/>[App.jsx]"]
end

subgraph group_api["API orchestration"]
  node_api_routes["Upload and ask routes<br/>[routes.py]"]
  node_schemas["Request and response schemas<br/>[schemas.py]"]
  node_pipeline["RAG pipeline<br/>[rag_pipeline.py]"]
end

subgraph group_ingestion["Document ingestion"]
  node_parser["Document parser<br/>[parser.py]"]
  node_cleaner["Text cleaner<br/>[cleaner.py]"]
  node_chunker["Page-aware chunker<br/>[chunker.py]"]
end

subgraph group_retrieval["Search and indexing"]
  node_embedder["BGE embedder<br/>[embedder.py]"]
  node_index_manager["FAISS index manager<br/>[index_manager.py]"]
  node_vector_store[("FAISS vector store<br/>[vector_store.py]")]
  node_bm25_manager["BM25 index manager<br/>[bm25_manager.py]"]
  node_bm25[("BM25 lexical store<br/>[bm25.py]")]
  node_hybrid["Hybrid RRF retrieval<br/>[hybrid.py]"]
  node_reranker["BGE reranker<br/>[reranker.py]"]
end

subgraph group_answer["Grounded answering"]
  node_prompt["Grounded prompt builder<br/>[prompt_builder.py]"]
  node_llm["Ollama local LLM<br/>[llm.py]"]
end

node_user(("Document user"))
node_ollama{{"Ollama service"}}

node_user -->|"uploads and asks"| node_web
node_web -->|"sends HTTP requests"| node_api_routes
node_api_routes -->|"validates and formats"| node_schemas
node_api_routes -->|"invokes indexing and ask"| node_pipeline
node_pipeline -->|"extracts document text"| node_parser
node_pipeline -->|"cleans page text"| node_cleaner
node_pipeline -->|"creates chunks"| node_chunker
node_pipeline -->|"embeds chunks and queries"| node_embedder
node_pipeline -->|"loads and saves dense index"| node_index_manager
node_pipeline -->|"loads and saves lexical index"| node_bm25_manager
node_pipeline -->|"replaces and adds chunks"| node_vector_store
node_pipeline -->|"replaces and adds chunks"| node_bm25
node_index_manager -->|"embeds indexed text"| node_embedder
node_index_manager -->|"builds dense index"| node_vector_store
node_bm25_manager -->|"builds lexical index"| node_bm25
node_pipeline -->|"retrieves candidates"| node_hybrid
node_hybrid -->|"searches semantic index"| node_vector_store
node_hybrid -->|"searches lexical index"| node_bm25
node_pipeline -->|"reranks candidates"| node_reranker
node_pipeline -->|"builds answer prompt"| node_prompt
node_pipeline -->|"generates answer"| node_llm
node_llm -.->|"requests local generation"| node_ollama
node_api_routes -->|"returns answer and citations"| node_web
node_prompt -.->|"uses retrieved chunk type"| node_vector_store

click node_web "https://github.com/thegreatpratyush/rag/blob/main/frontend/src/App.jsx"
click node_api_routes "https://github.com/thegreatpratyush/rag/blob/main/src/api/routes.py"
click node_schemas "https://github.com/thegreatpratyush/rag/blob/main/src/api/schemas.py"
click node_pipeline "https://github.com/thegreatpratyush/rag/blob/main/src/pipeline/rag_pipeline.py"
click node_parser "https://github.com/thegreatpratyush/rag/blob/main/src/ingestion/parser.py"
click node_cleaner "https://github.com/thegreatpratyush/rag/blob/main/src/ingestion/cleaner.py"
click node_chunker "https://github.com/thegreatpratyush/rag/blob/main/src/ingestion/chunker.py"
click node_embedder "https://github.com/thegreatpratyush/rag/blob/main/src/embeddings/embedder.py"
click node_index_manager "https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/index_manager.py"
click node_vector_store "https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/vector_store.py"
click node_bm25_manager "https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/bm25_manager.py"
click node_bm25 "https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/bm25.py"
click node_hybrid "https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/hybrid.py"
click node_reranker "https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/reranker.py"
click node_prompt "https://github.com/thegreatpratyush/rag/blob/main/src/generation/prompt_builder.py"
click node_llm "https://github.com/thegreatpratyush/rag/blob/main/src/generation/llm.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web,node_user toneBlue
class node_api_routes,node_schemas,node_pipeline toneAmber
class node_parser,node_cleaner,node_chunker toneMint
class node_embedder,node_index_manager,node_vector_store,node_bm25_manager,node_bm25,node_hybrid,node_reranker toneRose
class node_prompt,node_llm,node_ollama toneIndigo
```

## Components

### User interface

- [RAG interface](https://github.com/thegreatpratyush/rag/blob/main/frontend/src/App.jsx): `frontend/src/App.jsx`

### API orchestration

- [Upload and ask routes](https://github.com/thegreatpratyush/rag/blob/main/src/api/routes.py): `src/api/routes.py`
- [Request and response schemas](https://github.com/thegreatpratyush/rag/blob/main/src/api/schemas.py): `src/api/schemas.py`
- [RAG pipeline](https://github.com/thegreatpratyush/rag/blob/main/src/pipeline/rag_pipeline.py): `src/pipeline/rag_pipeline.py`

### Document ingestion

- [Document parser](https://github.com/thegreatpratyush/rag/blob/main/src/ingestion/parser.py): `src/ingestion/parser.py`
- [Text cleaner](https://github.com/thegreatpratyush/rag/blob/main/src/ingestion/cleaner.py): `src/ingestion/cleaner.py`
- [Page-aware chunker](https://github.com/thegreatpratyush/rag/blob/main/src/ingestion/chunker.py): `src/ingestion/chunker.py`

### Search and indexing

- [BGE embedder](https://github.com/thegreatpratyush/rag/blob/main/src/embeddings/embedder.py): `src/embeddings/embedder.py`
- [FAISS index manager](https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/index_manager.py): `src/retrieval/index_manager.py`
- [FAISS vector store](https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/vector_store.py): `src/retrieval/vector_store.py`
- [BM25 index manager](https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/bm25_manager.py): `src/retrieval/bm25_manager.py`
- [BM25 lexical store](https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/bm25.py): `src/retrieval/bm25.py`
- [Hybrid RRF retrieval](https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/hybrid.py): `src/retrieval/hybrid.py`
- [BGE reranker](https://github.com/thegreatpratyush/rag/blob/main/src/retrieval/reranker.py): `src/retrieval/reranker.py`

### Grounded answering

- [Grounded prompt builder](https://github.com/thegreatpratyush/rag/blob/main/src/generation/prompt_builder.py): `src/generation/prompt_builder.py`
- [Ollama local LLM](https://github.com/thegreatpratyush/rag/blob/main/src/generation/llm.py): `src/generation/llm.py`

### Other components

- **Document user**
- **Ollama service**

## Connections

- Document user → RAG interface: uploads and asks (evidence: `frontend/src/App.jsx`)
- RAG interface → Upload and ask routes: sends HTTP requests (evidence: `frontend/src/App.jsx`)
- Upload and ask routes → Request and response schemas: validates and formats (evidence: `src/api/routes.py`)
- Upload and ask routes → RAG pipeline: invokes indexing and ask (evidence: `src/api/routes.py`)
- RAG pipeline → Document parser: extracts document text (evidence: `src/pipeline/rag_pipeline.py`)
- RAG pipeline → Text cleaner: cleans page text (evidence: `src/pipeline/rag_pipeline.py`)
- RAG pipeline → Page-aware chunker: creates chunks (evidence: `src/pipeline/rag_pipeline.py`)
- RAG pipeline → BGE embedder: embeds chunks and queries (evidence: `src/pipeline/rag_pipeline.py`)
- RAG pipeline → FAISS index manager: loads and saves dense index (evidence: `src/pipeline/rag_pipeline.py`)
- RAG pipeline → BM25 index manager: loads and saves lexical index (evidence: `src/pipeline/rag_pipeline.py`)
- RAG pipeline → FAISS vector store: replaces and adds chunks (evidence: `src/pipeline/rag_pipeline.py`)
- RAG pipeline → BM25 lexical store: replaces and adds chunks (evidence: `src/pipeline/rag_pipeline.py`)
- FAISS index manager → BGE embedder: embeds indexed text (evidence: `src/retrieval/index_manager.py`)
- FAISS index manager → FAISS vector store: builds dense index (evidence: `src/retrieval/index_manager.py`)
- BM25 index manager → BM25 lexical store: builds lexical index (evidence: `src/retrieval/bm25_manager.py`)
- RAG pipeline → Hybrid RRF retrieval: retrieves candidates (evidence: `src/pipeline/rag_pipeline.py`)
- Hybrid RRF retrieval → FAISS vector store: searches semantic index (evidence: `src/retrieval/hybrid.py`)
- Hybrid RRF retrieval → BM25 lexical store: searches lexical index (evidence: `src/retrieval/hybrid.py`)
- RAG pipeline → BGE reranker: reranks candidates (evidence: `src/pipeline/rag_pipeline.py`)
- RAG pipeline → Grounded prompt builder: builds answer prompt (evidence: `src/pipeline/rag_pipeline.py`)
- RAG pipeline → Ollama local LLM: generates answer (evidence: `src/pipeline/rag_pipeline.py`)
- Ollama local LLM → Ollama service: requests local generation
- Upload and ask routes → RAG interface: returns answer and citations (evidence: `src/api/routes.py`)
- Grounded prompt builder → FAISS vector store: uses retrieved chunk type (evidence: `src/generation/prompt_builder.py`)
