How tanishshetty495/rag-from-scratch works
RAG From Scratch is a Flask demo that compares GPT-2 answers generated with and without retrieved context. The user submits a question and selects BERT or DPR; the API delegates the comparison to the pipeline, which retrieves passages and generates both answers for display. Retrieval builds or loads a cached FAISS index from the corpus. The README also documents a BERT-vs-DPR retrieval evaluation. The graph follows shown calls and documented behavior; model internals and unsampled frontend markup are not expanded.
Main entry points: retrieval.py Retriever · pipeline.py RAG comparison · evaluate_retrieval.py Retrieval evaluation · app.py Flask API
Components 15
Web interface
- Question UI
static/script.js - Page markup
templates/index.html
Application flow
- Flask API
app.py - RAG comparison
src/pipeline.py - Retrieval evaluation
scripts/evaluate_retrieval.py
Retrieval engine
- Retriever
src/retrieval.py - Text chunking
src/chunking.py - Knowledge corpus
data/corpus.txt - FAISS index
src/index_store.py - Index cache
cache - RAG configuration
src/config.py
Model backends
- BERT embedder
src/embeddings_bert.py - DPR embedder
src/embeddings_dpr.py - GPT-2 generator
src/generation.py
Other components
- Questioner
Connections 18
- Questioner → Question UI: asks question
static/script.js - Question UI → Flask API: posts question
static/script.js - Flask API → Page markup: renders page
app.py - Flask API → RAG comparison: runs comparison
app.py - RAG comparison → Retriever: creates retriever
src/pipeline.py - RAG comparison → BERT embedder: loads backend
src/pipeline.py - RAG comparison → DPR embedder: loads backend
src/pipeline.py - RAG comparison → GPT-2 generator: generates answers
src/pipeline.py - RAG comparison → Question UI: returns comparison
static/script.js - Retriever → Text chunking: splits corpus
src/retrieval.py - Retriever → Knowledge corpus: reads corpus
src/retrieval.py - Retriever → FAISS index: builds or searches
src/retrieval.py - FAISS index → Index cache: saves or loads
src/index_store.py - Retriever → BERT embedder: encodes and searches
- Retriever → DPR embedder: encodes and searches
- Retrieval evaluation → BERT embedder: evaluates backend
scripts/evaluate_retrieval.py - Retrieval evaluation → DPR embedder: evaluates backend
scripts/evaluate_retrieval.py - Retrieval evaluation → Retriever: measures retrieval
scripts/evaluate_retrieval.py