# abhinav-kimothi/a-simple-guide-to-rag architecture

> Architecture diagram of the GitHub repository abhinav-kimothi/a-simple-guide-to-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abhinav-kimothi/a-simple-guide-to-rag
- Repository: https://github.com/abhinav-kimothi/a-simple-guide-to-rag

## Overview

##### Project type and purpose
- This is an educational/demo repository accompanying the book “A Simple Guide to Retrieval Augmented Generation.”
- It is not a deployed service but a collection of Jupyter notebooks illustrating the end-to-end RAG workflow: indexing (Chapter-03), generation (Chapter-04), evaluation (Chapter-05), and advanced RAG strategies (Chapter-06).

##### File structure
- Top-level:
- Assets/Images – static illustrations
- Chapters/Chapter-03…Chapter-06 – each contains one or more notebooks (indexing, generation, evaluators, advanced techniques)
- README.md, example_dot_env, requirements.txt – project metadata, environment setup
- No “frontend” or “backend” folders because this is a data/ML pipeline demo.
- Dependencies are managed via requirements.txt; notebook code uses OpenAI, LangChain, FAISS, text-splitter, TF-IDF/BM25, and RAGAS.

##### README for additional insights
- The README describes three pipelines: indexing, generation, evaluation, and notes advanced RAG optimizations.
- Key technologies: AsyncHtmlLoader, Html2TextTransformer, RecursiveCharacterTextSplitter, OpenAI embeddings & GPT-4o, FAISS vector index, TF-IDF/BM25 retrievers, RAGAS, LangChain Benchmarks.

a. Data Ingestion & Preprocessing (Chapter-03):
– HTML Loader → Text Transformer → Chunker
b. Embedding & Indexing (Chapter-03):
– OpenAI Embedding API → FAISS Vector Store
c. Retrieval & Augmented Generation (Chapter-04):
– Query input → FAISS Similarity Search (or TF-IDF/BM25) → Retrieved chunks → Prompt + GPT-4o → Response
d. Evaluation (Chapter-05):
– RAGAS framework + LangChain Benchmarks → Metrics (accuracy, relevance, faithfulness)
e. Advanced RAG Strategies (Chapter-06):
– Index/query optimizations, retrieval strategies, compression

##### Architectural patterns & design principles
- Modular pipeline stages (Ingest → Embed → Retrieve → Generate → Evaluate)
- Separation of concerns: each notebook/module handles one stage
- Plug-and-play retrievers (swap FAISS, TF-IDF, BM25)
- Evaluation as a separate, reproducible module

##### Capturing essential elements
- Interchangeability of retrievers (FAISS vs TF-IDF/BM25).
- Evaluation as a feedback loop that can inform index or retrieval adjustments.
- Environment/configuration dependencies (.env, requirements.txt).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Ingestion & Preprocessing"
        Preproc["HTML Loader → Text Transformer → Chunker"]:::flowproc
    end

    Preproc -->|"documents"| EmbeddingExt
    EmbeddingExt["OpenAI Embedding API"]:::external
    EmbeddingExt --> IndexStore
    IndexStore["FAISS Vector Store"]:::storage

    User["User Query"]:::config
    User --> RetrieverFAISS
    User --> TFIDF

    RetrieverFAISS["FAISS Similarity Search"]:::flowproc
    TFIDF["TF-IDF/BM25 Retriever"]:::flowproc

    RetrieverFAISS --> Generator
    TFIDF --> Generator
    Generator["Prompt + GPT-4o Model"]:::external
    Generator --> UserResponse["Generated Response"]:::flowproc

    UserResponse --> EvalFramework
    EvalFramework["Evaluation Framework (RAGAS + LangChain Benchmarks)"]:::evaluation
    EvalFramework --> EvalNB
    EvalNB["Evaluation Pipeline Notebooks"]:::evaluation
    EvalNB -->|"feedback"| Preproc

    subgraph "Advanced Strategies"
        Adv["Advanced RAG Strategies"]:::flowproc
    end
    Adv --> RetrieverFAISS
    Adv --> EmbeddingExt

    subgraph "Project Assets & Configurations"
        Assets["Assets/Images"]:::assets
        Env[".env Template"]:::config
        Reqs["requirements.txt"]:::config
        Readme["README.md"]:::config
    end

    click Preproc "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/blob/main/Chapters/Chapter-03/indexing_pipeline.ipynb"
    click IndexStore "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/blob/main/Chapters/Chapter-03/indexing_pipeline.ipynb"
    click RetrieverFAISS "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/blob/main/Chapters/Chapter-04/generation_pipeline.ipynb"
    click TFIDF "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/blob/main/Chapters/Chapter-04/xtra_tfidf_bm25_retriever.ipynb"
    click EvalFramework "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/blob/main/Chapters/Chapter-05/evaluators.py"
    click EvalNB "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/blob/main/Chapters/Chapter-05/rag_evaluations.ipynb"
    click Adv "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/blob/main/Chapters/Chapter-06/advanced_rag.ipynb"
    click Assets "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/tree/main/Assets/Images"
    click Env "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/tree/main/example_dot_env"
    click Reqs "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/blob/main/requirements.txt"
    click Readme "https://github.com/abhinav-kimothi/a-simple-guide-to-rag/blob/main/README.md"

    classDef flowproc fill:#D6EAF8,stroke:#1B4F72,color:#154360;
    classDef storage fill:#D5F5E3,stroke:#196F3D,color:#145A32;
    classDef external fill:#F9E79F,stroke:#B9770E,color:#7D6608;
    classDef evaluation fill:#E8DAEF,stroke:#7D3C98,color:#512E5F;
    classDef config fill:#F2F3F4,stroke:#566573,color:#2C3E50;
    classDef assets fill:#FADBD8,stroke:#943126,color:#641E16;
```
