How abhinav-kimothi/a-simple-guide-to-rag works
Project type and purpose
• This is an educational/demo repository accompanying the book “A Simple Guide to Retrieval Augmented Generation.”
• It is not a deployed service but a collection of Jupyter notebooks illustrating the end-to-end RAG workflow: indexing (Chapter-03), generation (Chapter-04), evaluation (Chapter-05), and advanced RAG strategies (Chapter-06).
File structure
• Top-level:
• Assets/Images – static illustrations
• Chapters/Chapter-03…Chapter-06 – each contains one or more notebooks (indexing, generation, evaluators, advanced techniques)
• README.md, example_dot_env, requirements.txt – project metadata, environment setup
• No “frontend” or “backend” folders because this is a data/ML pipeline demo.
• Dependencies are managed via requirements.txt; notebook code uses OpenAI, LangChain, FAISS, text-splitter, TF-IDF/BM25, and RAGAS.
README for additional insights
• The README describes three pipelines: indexing, generation, evaluation, and notes advanced RAG optimizations.
• Key technologies: AsyncHtmlLoader, Html2TextTransformer, RecursiveCharacterTextSplitter, OpenAI embeddings & GPT-4o, FAISS vector index, TF-IDF/BM25 retrievers, RAGAS, LangChain Benchmarks.
a. Data Ingestion & Preprocessing (Chapter-03):
– HTML Loader → Text Transformer → Chunker
b. Embedding & Indexing (Chapter-03):
– OpenAI Embedding API → FAISS Vector Store
c. Retrieval & Augmented Generation (Chapter-04):
– Query input → FAISS Similarity Search (or TF-IDF/BM25) → Retrieved chunks → Prompt + GPT-4o → Response
d. Evaluation (Chapter-05):
– RAGAS framework + LangChain Benchmarks → Metrics (accuracy, relevance, faithfulness)
e. Advanced RAG Strategies (Chapter-06):
– Index/query optimizations, retrieval strategies, compression
Architectural patterns & design principles
• Modular pipeline stages (Ingest → Embed → Retrieve → Generate → Evaluate)
• Separation of concerns: each notebook/module handles one stage
• Plug-and-play retrievers (swap FAISS, TF-IDF, BM25)
• Evaluation as a separate, reproducible module
Capturing essential elements
• Interchangeability of retrievers (FAISS vs TF-IDF/BM25).
• Evaluation as a feedback loop that can inform index or retrieval adjustments.
• Environment/configuration dependencies (.env, requirements.txt).