# fareedkhan-dev/complex-rag-guide architecture

> Architecture diagram of the GitHub repository fareedkhan-dev/complex-rag-guide, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fareedkhan-dev/complex-rag-guide
- Repository: https://github.com/fareedkhan-dev/complex-rag-guide

## Overview

This repository presents a document-question-answering RAG system built around a Harry Potter PDF. The documented workflow prepares and indexes the book, anonymizes a question, plans and executes retrieval tasks, replans as evidence arrives, and evaluates the final answer. The README also describes alternate chunking, subgraph-based retrieval and grounding, and answer-quality checks. The notebook is the documented pipeline implementation; helper_functions.py contains reusable text/PDF and evaluation utilities. The notebook source was not sampled, so component-level relationships below rely on the README’s workflow description.

Main entry points: [`RAG_pipeline.ipynb`](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb) (Task handler), [`Harry Potter - Book 1 - The Sorcerers Stone.pdf`](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/Harry%20Potter%20-%20Book%201%20-%20The%20Sorcerers%20Stone.pdf) (Book PDF), [`helper_functions.py`](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/helper_functions.py) (Text and PDF helpers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_prep["Document preparation"]
  node_book["Book PDF"]
  node_prep_pipeline["Clean and chunk<br/>[RAG_pipeline.ipynb]"]
  node_helpers["Text and PDF helpers"]
end

subgraph group_retrieve["Retrieval and grounding"]
  node_index[("Vectorization<br/>[RAG_pipeline.ipynb]")]
  node_retriever["Context retriever<br/>[RAG_pipeline.ipynb]"]
  node_subgraphs["Retrieval subgraphs<br/>[RAG_pipeline.ipynb]"]
  node_quote_tool["Quote retrieval<br/>[RAG_pipeline.ipynb]"]
  node_chunk_tool["Chunk retrieval<br/>[RAG_pipeline.ipynb]"]
  node_summary_tool["Chapter summaries<br/>[RAG_pipeline.ipynb]"]
  node_grounding["Grounding checks<br/>[RAG_pipeline.ipynb]"]
end

subgraph group_agent["Planning and answers"]
  node_anonymizer["Question anonymizer<br/>[RAG_pipeline.ipynb]"]
  node_planner["Planner<br/>[RAG_pipeline.ipynb]"]
  node_plan_tools["Plan breakdown<br/>[RAG_pipeline.ipynb]"]
  node_task_handler["Task handler<br/>[RAG_pipeline.ipynb]"]
  node_direct_answer["Direct answer<br/>[RAG_pipeline.ipynb]"]
  node_replanner["Replanner<br/>[RAG_pipeline.ipynb]"]
  node_final_answer["Answer synthesis<br/>[RAG_pipeline.ipynb]"]
  node_deanonymizer["Answer de-anonymizer<br/>[RAG_pipeline.ipynb]"]
end

subgraph group_eval["Evaluation"]
  node_ragas["RAGAS metrics<br/>[RAG_pipeline.ipynb]"]
  node_metric_helpers["Metric reporting"]
end

node_user(("Questioner"))
node_llm{{"LLM providers"}}

node_book -->|"loads and splits"| node_prep_pipeline
node_prep_pipeline -.->|"uses utilities"| node_helpers
node_prep_pipeline -->|"vectorizes"| node_index
node_index -->|"provides context"| node_retriever
node_retriever -.->|"supports focused retrieval"| node_subgraphs
node_user -->|"asks question"| node_anonymizer
node_anonymizer -->|"anonymizes input"| node_planner
node_planner -->|"plans with"| node_llm
node_planner -->|"hands off plan"| node_plan_tools
node_plan_tools -->|"creates tasks"| node_task_handler
node_task_handler -->|"selects tool"| node_quote_tool
node_task_handler -->|"selects tool"| node_chunk_tool
node_task_handler -->|"selects tool"| node_summary_tool
node_task_handler -->|"selects tool"| node_direct_answer
node_quote_tool -->|"retrieves quotes"| node_retriever
node_chunk_tool -->|"retrieves chunks"| node_retriever
node_summary_tool -->|"retrieves summaries"| node_retriever
node_task_handler -->|"submits progress"| node_replanner
node_replanner -->|"updates tasks"| node_task_handler
node_replanner -->|"checks answerability"| node_grounding
node_grounding -->|"permits synthesis"| node_final_answer
node_final_answer -->|"synthesizes with"| node_llm
node_final_answer -->|"restores names"| node_deanonymizer
node_deanonymizer -->|"evaluates response"| node_ragas
node_ragas -.->|"reports metrics"| node_metric_helpers
node_llm -.->|"returns plan"| node_planner
node_llm -.->|"returns response"| node_final_answer
node_ragas -.->|"delivers verified answer"| node_user

click node_book "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/Harry%20Potter%20-%20Book%201%20-%20The%20Sorcerers%20Stone.pdf"
click node_prep_pipeline "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_helpers "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/helper_functions.py"
click node_index "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_retriever "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_subgraphs "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_anonymizer "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_planner "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_plan_tools "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_task_handler "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_quote_tool "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_chunk_tool "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_summary_tool "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_direct_answer "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_replanner "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_grounding "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_final_answer "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_deanonymizer "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_ragas "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb"
click node_metric_helpers "https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/helper_functions.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_book,node_prep_pipeline,node_helpers toneBlue
class node_index,node_retriever,node_subgraphs,node_quote_tool,node_chunk_tool,node_summary_tool,node_grounding toneAmber
class node_anonymizer,node_planner,node_plan_tools,node_task_handler,node_direct_answer,node_replanner,node_final_answer,node_deanonymizer toneMint
class node_ragas,node_metric_helpers toneRose
class node_user,node_llm toneIndigo
```

## Components

### Document preparation

- [Book PDF](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/Harry%20Potter%20-%20Book%201%20-%20The%20Sorcerers%20Stone.pdf): `Harry Potter - Book 1 - The Sorcerers Stone.pdf`
- [Clean and chunk](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Text and PDF helpers](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/helper_functions.py): `helper_functions.py`

### Retrieval and grounding

- [Vectorization](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Context retriever](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Retrieval subgraphs](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Quote retrieval](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Chunk retrieval](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Chapter summaries](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Grounding checks](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`

### Planning and answers

- [Question anonymizer](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Planner](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Plan breakdown](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Task handler](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Direct answer](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Replanner](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Answer synthesis](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Answer de-anonymizer](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`

### Evaluation

- [RAGAS metrics](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/RAG_pipeline.ipynb): `RAG_pipeline.ipynb`
- [Metric reporting](https://github.com/fareedkhan-dev/complex-rag-guide/blob/main/helper_functions.py): `helper_functions.py`

### Other components

- **Questioner**
- **LLM providers**

## Connections

- Book PDF → Clean and chunk: loads and splits (evidence: `README.md`)
- Clean and chunk → Text and PDF helpers: uses utilities
- Clean and chunk → Vectorization: vectorizes (evidence: `README.md`)
- Vectorization → Context retriever: provides context (evidence: `README.md`)
- Context retriever → Retrieval subgraphs: supports focused retrieval (evidence: `README.md`)
- Questioner → Question anonymizer: asks question (evidence: `README.md`)
- Question anonymizer → Planner: anonymizes input (evidence: `README.md`)
- Planner → LLM providers: plans with (evidence: `README.md`)
- Planner → Plan breakdown: hands off plan (evidence: `README.md`)
- Plan breakdown → Task handler: creates tasks (evidence: `README.md`)
- Task handler → Quote retrieval: selects tool (evidence: `README.md`)
- Task handler → Chunk retrieval: selects tool (evidence: `README.md`)
- Task handler → Chapter summaries: selects tool (evidence: `README.md`)
- Task handler → Direct answer: selects tool (evidence: `README.md`)
- Quote retrieval → Context retriever: retrieves quotes (evidence: `README.md`)
- Chunk retrieval → Context retriever: retrieves chunks (evidence: `README.md`)
- Chapter summaries → Context retriever: retrieves summaries (evidence: `README.md`)
- Task handler → Replanner: submits progress (evidence: `README.md`)
- Replanner → Task handler: updates tasks (evidence: `README.md`)
- Replanner → Grounding checks: checks answerability (evidence: `README.md`)
- Grounding checks → Answer synthesis: permits synthesis (evidence: `README.md`)
- Answer synthesis → LLM providers: synthesizes with (evidence: `README.md`)
- Answer synthesis → Answer de-anonymizer: restores names (evidence: `README.md`)
- Answer de-anonymizer → RAGAS metrics: evaluates response (evidence: `README.md`)
- RAGAS metrics → Metric reporting: reports metrics
- LLM providers → Planner: returns plan
- LLM providers → Answer synthesis: returns response
- RAGAS metrics → Questioner: delivers verified answer (evidence: `README.md`)
