# tanishshetty495/rag-from-scratch architecture

> Architecture diagram of the GitHub repository tanishshetty495/rag-from-scratch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tanishshetty495/rag-from-scratch
- Repository: https://github.com/tanishshetty495/rag-from-scratch

## Overview

RAG From Scratch is a Flask demo that compares GPT-2 answers generated with and without retrieved context. The user submits a question and selects BERT or DPR; the API delegates the comparison to the pipeline, which retrieves passages and generates both answers for display. Retrieval builds or loads a cached FAISS index from the corpus. The README also documents a BERT-vs-DPR retrieval evaluation. The graph follows shown calls and documented behavior; model internals and unsampled frontend markup are not expanded.

Main entry points: [`src/retrieval.py`](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/retrieval.py) (Retriever), [`src/pipeline.py`](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/pipeline.py) (RAG comparison), [`scripts/evaluate_retrieval.py`](https://github.com/tanishshetty495/rag-from-scratch/blob/main/scripts/evaluate_retrieval.py) (Retrieval evaluation), [`app.py`](https://github.com/tanishshetty495/rag-from-scratch/blob/main/app.py) (Flask API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["Web interface"]
  node_browser["Question UI<br/>[script.js]"]
  node_page["Page markup<br/>[index.html]"]
end

subgraph group_application["Application flow"]
  node_flask["Flask API<br/>[app.py]"]
  node_pipeline["RAG comparison<br/>[pipeline.py]"]
  node_evaluation["Retrieval evaluation"]
end

subgraph group_retrieval["Retrieval engine"]
  node_retriever["Retriever<br/>[retrieval.py]"]
  node_chunker["Text chunking<br/>[chunking.py]"]
  node_corpus["Knowledge corpus<br/>[corpus.txt]"]
  node_vector_index["FAISS index<br/>[index_store.py]"]
  node_index_cache[("Index cache")]
  node_config["RAG configuration<br/>[config.py]"]
end

subgraph group_models["Model backends"]
  node_bert["BERT embedder<br/>[embeddings_bert.py]"]
  node_dpr["DPR embedder<br/>[embeddings_dpr.py]"]
  node_generator["GPT-2 generator<br/>[generation.py]"]
end

node_user(("Questioner"))

node_user -->|"asks question"| node_browser
node_browser -->|"posts question"| node_flask
node_flask -->|"renders page"| node_page
node_flask -->|"runs comparison"| node_pipeline
node_pipeline -->|"creates retriever"| node_retriever
node_pipeline -->|"loads backend"| node_bert
node_pipeline -->|"loads backend"| node_dpr
node_pipeline -->|"generates answers"| node_generator
node_pipeline -->|"returns comparison"| node_browser
node_retriever -->|"splits corpus"| node_chunker
node_retriever -->|"reads corpus"| node_corpus
node_retriever -->|"builds or searches"| node_vector_index
node_vector_index -->|"saves or loads"| node_index_cache
node_retriever -.->|"encodes and searches"| node_bert
node_retriever -.->|"encodes and searches"| node_dpr
node_evaluation -->|"evaluates backend"| node_bert
node_evaluation -->|"evaluates backend"| node_dpr
node_evaluation -->|"measures retrieval"| node_retriever

click node_browser "https://github.com/tanishshetty495/rag-from-scratch/blob/main/static/script.js"
click node_page "https://github.com/tanishshetty495/rag-from-scratch/blob/main/templates/index.html"
click node_flask "https://github.com/tanishshetty495/rag-from-scratch/blob/main/app.py"
click node_pipeline "https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/pipeline.py"
click node_evaluation "https://github.com/tanishshetty495/rag-from-scratch/blob/main/scripts/evaluate_retrieval.py"
click node_retriever "https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/retrieval.py"
click node_chunker "https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/chunking.py"
click node_corpus "https://github.com/tanishshetty495/rag-from-scratch/blob/main/data/corpus.txt"
click node_vector_index "https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/index_store.py"
click node_index_cache "https://github.com/tanishshetty495/rag-from-scratch/tree/main/cache"
click node_config "https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/config.py"
click node_bert "https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/embeddings_bert.py"
click node_dpr "https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/embeddings_dpr.py"
click node_generator "https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/generation.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser,node_page toneBlue
class node_flask,node_pipeline,node_evaluation toneAmber
class node_retriever,node_chunker,node_corpus,node_vector_index,node_index_cache,node_config toneMint
class node_bert,node_dpr,node_generator toneRose
class node_user toneIndigo
```

## Components

### Web interface

- [Question UI](https://github.com/tanishshetty495/rag-from-scratch/blob/main/static/script.js): `static/script.js`
- [Page markup](https://github.com/tanishshetty495/rag-from-scratch/blob/main/templates/index.html): `templates/index.html`

### Application flow

- [Flask API](https://github.com/tanishshetty495/rag-from-scratch/blob/main/app.py): `app.py`
- [RAG comparison](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/pipeline.py): `src/pipeline.py`
- [Retrieval evaluation](https://github.com/tanishshetty495/rag-from-scratch/blob/main/scripts/evaluate_retrieval.py): `scripts/evaluate_retrieval.py`

### Retrieval engine

- [Retriever](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/retrieval.py): `src/retrieval.py`
- [Text chunking](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/chunking.py): `src/chunking.py`
- [Knowledge corpus](https://github.com/tanishshetty495/rag-from-scratch/blob/main/data/corpus.txt): `data/corpus.txt`
- [FAISS index](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/index_store.py): `src/index_store.py`
- [Index cache](https://github.com/tanishshetty495/rag-from-scratch/tree/main/cache): `cache`
- [RAG configuration](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/config.py): `src/config.py`

### Model backends

- [BERT embedder](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/embeddings_bert.py): `src/embeddings_bert.py`
- [DPR embedder](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/embeddings_dpr.py): `src/embeddings_dpr.py`
- [GPT-2 generator](https://github.com/tanishshetty495/rag-from-scratch/blob/main/src/generation.py): `src/generation.py`

### Other components

- **Questioner**

## Connections

- Questioner → Question UI: asks question (evidence: `static/script.js`)
- Question UI → Flask API: posts question (evidence: `static/script.js`)
- Flask API → Page markup: renders page (evidence: `app.py`)
- Flask API → RAG comparison: runs comparison (evidence: `app.py`)
- RAG comparison → Retriever: creates retriever (evidence: `src/pipeline.py`)
- RAG comparison → BERT embedder: loads backend (evidence: `src/pipeline.py`)
- RAG comparison → DPR embedder: loads backend (evidence: `src/pipeline.py`)
- RAG comparison → GPT-2 generator: generates answers (evidence: `src/pipeline.py`)
- RAG comparison → Question UI: returns comparison (evidence: `static/script.js`)
- Retriever → Text chunking: splits corpus (evidence: `src/retrieval.py`)
- Retriever → Knowledge corpus: reads corpus (evidence: `src/retrieval.py`)
- Retriever → FAISS index: builds or searches (evidence: `src/retrieval.py`)
- FAISS index → Index cache: saves or loads (evidence: `src/index_store.py`)
- Retriever → BERT embedder: encodes and searches
- Retriever → DPR embedder: encodes and searches
- Retrieval evaluation → BERT embedder: evaluates backend (evidence: `scripts/evaluate_retrieval.py`)
- Retrieval evaluation → DPR embedder: evaluates backend (evidence: `scripts/evaluate_retrieval.py`)
- Retrieval evaluation → Retriever: measures retrieval (evidence: `scripts/evaluate_retrieval.py`)
