# mghayaz/book_rag architecture

> Architecture diagram of the GitHub repository mghayaz/book_rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mghayaz/book_rag
- Repository: https://github.com/mghayaz/book_rag

## Overview

This repository implements book-question retrieval: PDFs are extracted and chunked, chunks are embedded into a persistent Chroma store, and queries can retrieve candidates and rerank them. A separate evaluation script runs retrieval against a golden dataset using contextual recall and precision metrics. There is no README or application/API layer shown, so the user-facing entry point and any serving workflow are uncertain; the graph focuses on the evidenced retrieval and evaluation paths.

Main entry points: [`src/retriever.py`](https://github.com/mghayaz/book_rag/blob/main/src/retriever.py) (Vector retrieval), [`src/reranker.py`](https://github.com/mghayaz/book_rag/blob/main/src/reranker.py) (Cross-encoder reranker), [`evals/eval_retriever.py`](https://github.com/mghayaz/book_rag/blob/main/evals/eval_retriever.py) (Evaluation runner), [`chorma_book_store`](https://github.com/mghayaz/book_rag/tree/main/chorma_book_store) (Persistent vector store)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Book ingestion"]
  node_pdf_books["Book PDFs"]
  node_pdf_loader["PDF extraction<br/>[retriever.py]"]
  node_chunk_embed["Chunk and embed<br/>[retriever.py]"]
  node_chroma_store[("Persistent vector store")]
end

subgraph group_retrieval["Search and ranking"]
  node_retriever["Vector retrieval<br/>[retriever.py]"]
  node_reranker["Cross-encoder reranker<br/>[reranker.py]"]
end

subgraph group_configuration["Configuration"]
  node_settings["Runtime settings<br/>[config.py]"]
end

subgraph group_evaluation["Retrieval evaluation"]
  node_eval_runner["Evaluation runner<br/>[eval_retriever.py]"]
  node_golden_dataset["Golden queries<br/>[dataset.json]"]
  node_eval_metrics["Contextual metrics<br/>[eval_retriever.py]"]
end

node_query_user(("Query initiator"))
node_openai_embeddings{{"OpenAI embeddings"}}
node_cross_encoder_model{{"Cross-encoder model"}}

node_pdf_books -->|"reads PDFs"| node_pdf_loader
node_pdf_loader -->|"passes documents"| node_chunk_embed
node_chunk_embed -->|"embeds chunks"| node_openai_embeddings
node_chunk_embed -->|"stores chunks"| node_chroma_store
node_retriever -->|"opens store"| node_chroma_store
node_retriever -->|"reads settings"| node_settings
node_retriever -->|"loads PDFs"| node_pdf_loader
node_retriever -->|"chunks documents"| node_chunk_embed
node_retriever -->|"configures embeddings"| node_openai_embeddings
node_query_user -.->|"submits query"| node_reranker
node_reranker -->|"loads store"| node_retriever
node_reranker -->|"searches candidates"| node_chroma_store
node_reranker -->|"scores pairs"| node_cross_encoder_model
node_eval_runner -->|"loads cases"| node_golden_dataset
node_eval_runner -->|"invokes retrieval"| node_reranker
node_eval_runner -->|"evaluates results"| node_eval_metrics
node_eval_runner -->|"reads settings"| node_settings

click node_pdf_books "https://github.com/mghayaz/book_rag/tree/main/Books"
click node_pdf_loader "https://github.com/mghayaz/book_rag/blob/main/src/retriever.py"
click node_chunk_embed "https://github.com/mghayaz/book_rag/blob/main/src/retriever.py"
click node_chroma_store "https://github.com/mghayaz/book_rag/tree/main/chorma_book_store"
click node_retriever "https://github.com/mghayaz/book_rag/blob/main/src/retriever.py"
click node_reranker "https://github.com/mghayaz/book_rag/blob/main/src/reranker.py"
click node_settings "https://github.com/mghayaz/book_rag/blob/main/core/config.py"
click node_eval_runner "https://github.com/mghayaz/book_rag/blob/main/evals/eval_retriever.py"
click node_golden_dataset "https://github.com/mghayaz/book_rag/blob/main/goldens/dataset.json"
click node_eval_metrics "https://github.com/mghayaz/book_rag/blob/main/evals/eval_retriever.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pdf_books,node_pdf_loader,node_chunk_embed,node_chroma_store toneBlue
class node_retriever,node_reranker toneAmber
class node_settings toneMint
class node_eval_runner,node_golden_dataset,node_eval_metrics toneRose
class node_query_user,node_openai_embeddings,node_cross_encoder_model toneIndigo
```

## Components

### Book ingestion

- [Book PDFs](https://github.com/mghayaz/book_rag/tree/main/Books): `Books`
- [PDF extraction](https://github.com/mghayaz/book_rag/blob/main/src/retriever.py): `src/retriever.py`
- [Chunk and embed](https://github.com/mghayaz/book_rag/blob/main/src/retriever.py): `src/retriever.py`
- [Persistent vector store](https://github.com/mghayaz/book_rag/tree/main/chorma_book_store): `chorma_book_store`

### Search and ranking

- [Vector retrieval](https://github.com/mghayaz/book_rag/blob/main/src/retriever.py): `src/retriever.py`
- [Cross-encoder reranker](https://github.com/mghayaz/book_rag/blob/main/src/reranker.py): `src/reranker.py`

### Configuration

- [Runtime settings](https://github.com/mghayaz/book_rag/blob/main/core/config.py): `core/config.py`

### Retrieval evaluation

- [Evaluation runner](https://github.com/mghayaz/book_rag/blob/main/evals/eval_retriever.py): `evals/eval_retriever.py`
- [Golden queries](https://github.com/mghayaz/book_rag/blob/main/goldens/dataset.json): `goldens/dataset.json`
- [Contextual metrics](https://github.com/mghayaz/book_rag/blob/main/evals/eval_retriever.py): `evals/eval_retriever.py`

### Other components

- **Query initiator**
- **OpenAI embeddings**
- **Cross-encoder model**

## Connections

- Book PDFs → PDF extraction: reads PDFs (evidence: `src/retriever.py`)
- PDF extraction → Chunk and embed: passes documents (evidence: `src/retriever.py`)
- Chunk and embed → OpenAI embeddings: embeds chunks (evidence: `src/retriever.py`)
- Chunk and embed → Persistent vector store: stores chunks (evidence: `src/retriever.py`)
- Vector retrieval → Persistent vector store: opens store (evidence: `src/retriever.py`)
- Vector retrieval → Runtime settings: reads settings (evidence: `src/retriever.py`)
- Vector retrieval → PDF extraction: loads PDFs (evidence: `src/retriever.py`)
- Vector retrieval → Chunk and embed: chunks documents (evidence: `src/retriever.py`)
- Vector retrieval → OpenAI embeddings: configures embeddings (evidence: `src/retriever.py`)
- Query initiator → Cross-encoder reranker: submits query
- Cross-encoder reranker → Vector retrieval: loads store (evidence: `src/reranker.py`)
- Cross-encoder reranker → Persistent vector store: searches candidates (evidence: `src/reranker.py`)
- Cross-encoder reranker → Cross-encoder model: scores pairs (evidence: `src/reranker.py`)
- Evaluation runner → Golden queries: loads cases (evidence: `evals/eval_retriever.py`)
- Evaluation runner → Cross-encoder reranker: invokes retrieval (evidence: `evals/eval_retriever.py`)
- Evaluation runner → Contextual metrics: evaluates results (evidence: `evals/eval_retriever.py`)
- Evaluation runner → Runtime settings: reads settings (evidence: `evals/eval_retriever.py`)
