# prasad-j155/rag-automated-evaluation-pipeline architecture

> Architecture diagram of the GitHub repository prasad-j155/rag-automated-evaluation-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/prasad-j155/rag-automated-evaluation-pipeline
- Repository: https://github.com/prasad-j155/rag-automated-evaluation-pipeline

## Overview

##### Project type and purpose
- This is a full-stack Python application combining a batch pipeline and a web dashboard.
- Purpose: build a Hybrid Retrieval-Augmented Generation (RAG) system that
- Ingests and indexes web content (fixed + dynamic URLs)
- Performs sparse (BM25) and dense (FAISS) retrieval
- Fuses results via Reciprocal Rank Fusion (RRF)
- Generates answers using an LLM (google/flan-t5-base)
- Runs an automated evaluation suite (MRR, BERTScore)
- Exposes results via Streamlit UI

##### File structure
- Top-level scripts:
- run_pipeline.py – end-to-end orchestration of data gen, indexing, question gen, retrieval, generation, evaluation, report export
- app.py – Streamlit front end for interactive querying and visualization
- src/ modules:
- config.py – centralize all paths, hyperparameters (RRF k-value, index names, report output folder)
- data_loader.py – fetch fixed_urls.json + Wikipedia URLs, build corpus
- rag_engine.py – wrap sparse search (BM25Okapi), dense search (all-MiniLM-L6-v2 + FAISS), run RRF fusion
- evaluation.py – question generation, run ablation studies (Dense vs Sparse vs Hybrid), compute MRR and BERTScore, save charts/CSVs
- Requirements.txt – Python dependencies (faiss-cpu, rank-bm25, sentence-transformers, transformers, streamlit, bert_score)

##### Extract architecture notes from the README
- Data flows from “Data Source” → “Retrieval Engine” → “LLM Generator” → “Evaluation”
- Two retrieval subcomponents (sparse vs dense) feed into a fusion layer
- A single command (run_pipeline.py) drives the end-to-end flow
- app.py gives a live dashboard to query the same engine and view saved reports

a. Main components:
1. Data Sources
– Fixed URLs (local JSON)
– Dynamic URLs (Wikipedia API)
2. Data Loader
3. Indexing layer
– Sparse index (BM25Okapi)
– Dense index (Sentence-Transformer embeddings + FAISS)
4. Retrieval Engine
– BM25 search
– FAISS search
5. Fusion Module (Reciprocal Rank Fusion)
6. LLM Generator (google/flan-t5-base)
7. Evaluation Suite
– Question generator
– Metric calculators (MRR, BERTScore)
– Report exporter (charts + CSV into reports/)
8. Web Dashboard (Streamlit)
- Data Sources → Data Loader → Indexing layer
- Indexing layer → Retrieval Engine (two parallel branches: sparse and dense)
- Both retrieval outputs → Fusion Module → merged result
- Merged result + context → LLM Generator → answers
- Answers + ground-truth questions → Evaluation Suite → metrics + reports
- Web Dashboard queries Retrieval Engine / Fusion / Generator on demand
- Reports folder is a storage artifact
c. Architectural patterns and design principles:
– Modular service layers: ingestion, indexing, retrieval, fusion, generation, evaluation, presentation
– Pipeline orchestration script (run_pipeline.py) implements a linear DAG of tasks
– Single monolithic codebase with clear separation by Python module
d. Technologies/frameworks:
– BM25Okapi from rank-bm25
– SentenceTransformers for embedding generation
– FAISS vector store for fast nearest-neighbor search
– Reciprocal Rank Fusion (RRF) algorithm k=60
– Transformers LLM (google/flan-t5-base) for generation
– bert_score library for BERTScore
– Streamlit for UI dashboard
– Python 3.8+

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Configuration
    subgraph "Configuration"
        CONFIG["Configuration Module"]:::config
    end

    %% Ingestion & Indexing Layer
    subgraph "Ingestion & Indexing" 
        DS_FIXED["Fixed URLs (local JSON)"]:::external
        DS_WIKI["Wikipedia API"]:::external
        DL["Data Loader"]:::ingestion
        IDX_SPARSE["Sparse Index (BM25)"]:::indexing
        IDX_DENSE["Dense Index (FAISS)"]:::indexing
    end

    %% Retrieval & Fusion Layer
    subgraph "Retrieval & Fusion"
        RET_SPARSE["BM25 Search"]:::retrieval
        RET_DENSE["FAISS Search"]:::retrieval
        FUSION["RRF Fusion Module"]:::fusion
    end

    %% Generation & Evaluation Layer
    subgraph "Generation & Evaluation"
        LLM["LLM Generator\n(google/flan-t5-base)"]:::generation
        QGEN["Question Generator"]:::evaluation
        METRICS["Metric Calculator\n(MRR & BERTScore)"]:::evaluation
        EXPORT["Report Exporter\n(charts + CSV)"]:::evaluation
        REPORTS["reports/"]:::storage
    end

    %% UI & Orchestration
    subgraph "UI & Orchestration"
        BATCH["Batch Pipeline\n(run_pipeline.py)"]:::orchestration
        UI["Streamlit Dashboard\n(app.py)"]:::ui
    end

    %% Data Flow
    DS_FIXED --> DL
    DS_WIKI --> DL
    CONFIG --> DL

    DL --> IDX_SPARSE
    DL --> IDX_DENSE
    CONFIG --> IDX_SPARSE
    CONFIG --> IDX_DENSE

    IDX_SPARSE --> RET_SPARSE
    IDX_DENSE --> RET_DENSE

    RET_SPARSE --> FUSION
    RET_DENSE --> FUSION
    CONFIG --> FUSION

    FUSION --> LLM
    CONFIG --> LLM

    LLM --> QGEN
    QGEN --> METRICS
    METRICS --> EXPORT
    EXPORT --> REPORTS

    %% Batch flow
    BATCH --> DL
    BATCH --> RET_SPARSE
    BATCH --> RET_DENSE
    BATCH --> LLM
    BATCH --> QGEN
    EXPORT --> BATCH

    %% Interactive flow
    UI --> RET_SPARSE
    UI --> RET_DENSE
    UI --> FUSION
    UI --> LLM
    UI --> EXPORT

    %% Click Events
    click CONFIG "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/config.py"
    click DL "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/data_loader.py"
    click CONFIG "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/config.py"
    click FUSION "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/rag_engine.py"
    click RET_SPARSE "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/rag_engine.py"
    click RET_DENSE "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/rag_engine.py"
    click LLM "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/rag_engine.py"
    click QGEN "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/evaluation.py"
    click METRICS "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/evaluation.py"
    click EXPORT "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/src/evaluation.py"
    click BATCH "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/run_pipeline.py"
    click UI "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/app.py"
    click REPORTS "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/tree/main/reports/"
    click REQUIRE "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/requirements.txt"
    click README "https://github.com/prasad-j155/rag-automated-evaluation-pipeline/blob/main/README.md"

    %% Styles
    classDef config fill:#FFD580,stroke:#333,stroke-width:1px
    classDef ingestion fill:#FFF2CC,stroke:#333,stroke-width:1px
    classDef indexing fill:#FFE699,stroke:#333,stroke-width:1px
    classDef retrieval fill:#C9DAF8,stroke:#333,stroke-width:1px
    classDef fusion fill:#A4C2F4,stroke:#333,stroke-width:1px
    classDef generation fill:#D9EAD3,stroke:#333,stroke-width:1px
    classDef evaluation fill:#B6D7A8,stroke:#333,stroke-width:1px
    classDef ui fill:#EAD1DC,stroke:#333,stroke-width:1px
    classDef orchestration fill:#D5A6BD,stroke:#333,stroke-width:1px
    classDef external fill:#F4CCCC,stroke:#333,stroke-width:1px
    classDef storage fill:#EAD1DC,stroke:#333,stroke-width:1px
    classDef REQUIRE fill:#E0E0E0,stroke:#333,stroke-width:1px
    classDef README fill:#E0E0E0,stroke:#333,stroke-width:1px
```
