# trailblazer09/referencerover architecture

> Architecture diagram of the GitHub repository trailblazer09/referencerover, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/trailblazer09/referencerover
- Repository: https://github.com/trailblazer09/referencerover

## Overview

This is a command-line PDF RAG evaluation harness. A user supplies a question or requests an evaluation run; the CLI loads PDFs, indexes their text, retrieves relevant chunks, and passes the question and context to an LLM. Results include references, grounding status, and per-call token and latency metrics. The graph separates ingestion/indexing, question answering, and external services. Some implementation is unsampled, so ingestion and CLI wiring also rely on the README’s description; uncertain internal calls are omitted.

Main entry points: [`rag_evaluation.py`](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py) (CLI workflow), [`main.py`](https://github.com/trailblazer09/referencerover/blob/main/main.py) (CLI entry), [`pdfs`](https://github.com/trailblazer09/referencerover/tree/main/pdfs) (Source PDFs), [`evaluation_questions.json`](https://github.com/trailblazer09/referencerover/blob/main/evaluation_questions.json) (Evaluation questions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Ingestion and indexing"]
  node_pdfs["Source PDFs"]
  node_pdf_loader["PDF extraction<br/>[rag_evaluation.py]"]
  node_chunking["Text chunking<br/>[rag_evaluation.py]"]
  node_embedding["Embeddings<br/>[rag_evaluation.py]"]
  node_indexing["Document indexing<br/>[rag_evaluation.py]"]
end

subgraph group_answering["Answering and evaluation"]
  node_runtime["CLI workflow<br/>[rag_evaluation.py]"]
  node_questions["Evaluation questions"]
  node_evaluation["Question evaluation<br/>[rag_evaluation.py]"]
  node_search["Semantic search<br/>[rag_evaluation.py]"]
  node_grounding["Grounding check<br/>[rag_evaluation.py]"]
  node_llm_call["LLM call<br/>[rag_evaluation.py]"]
  node_results["Answers and metrics"]
end

subgraph group_services["External services"]
  node_qdrant[("Qdrant")]
  node_providers{{"Gemini or Groq"}}
end

node_user(("User"))
node_cli["CLI entry<br/>[main.py]"]

node_user -->|"runs"| node_cli
node_cli -->|"invokes"| node_runtime
node_runtime -->|"loads PDFs"| node_pdf_loader
node_pdfs -->|"provides files"| node_pdf_loader
node_pdf_loader -->|"extracts text"| node_chunking
node_runtime -->|"indexes documents"| node_indexing
node_indexing -->|"embeds chunks"| node_embedding
node_indexing -->|"writes vectors"| node_qdrant
node_runtime -->|"evaluates questions"| node_evaluation
node_questions -->|"supplies questions"| node_runtime
node_evaluation -->|"retrieves context"| node_search
node_search -->|"embeds query"| node_embedding
node_search -->|"searches vectors"| node_qdrant
node_evaluation -->|"requests answer"| node_llm_call
node_llm_call -->|"sends prompt"| node_providers
node_evaluation -->|"checks context"| node_grounding
node_evaluation -->|"returns answer and metrics"| node_results
node_runtime -->|"prints references"| node_results

click node_cli "https://github.com/trailblazer09/referencerover/blob/main/main.py"
click node_runtime "https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py"
click node_pdfs "https://github.com/trailblazer09/referencerover/tree/main/pdfs"
click node_pdf_loader "https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py"
click node_chunking "https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py"
click node_embedding "https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py"
click node_indexing "https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py"
click node_questions "https://github.com/trailblazer09/referencerover/blob/main/evaluation_questions.json"
click node_evaluation "https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py"
click node_search "https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py"
click node_grounding "https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py"
click node_llm_call "https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pdfs,node_pdf_loader,node_chunking,node_embedding,node_indexing,node_user toneBlue
class node_runtime,node_questions,node_evaluation,node_search,node_grounding,node_llm_call,node_results toneAmber
class node_qdrant,node_providers toneMint
class node_cli toneTeal
```

## Components

### Ingestion and indexing

- [Source PDFs](https://github.com/trailblazer09/referencerover/tree/main/pdfs): `pdfs`
- [PDF extraction](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py): `rag_evaluation.py`
- [Text chunking](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py): `rag_evaluation.py`
- [Embeddings](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py): `rag_evaluation.py`
- [Document indexing](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py): `rag_evaluation.py`

### Answering and evaluation

- [CLI workflow](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py): `rag_evaluation.py`
- [Evaluation questions](https://github.com/trailblazer09/referencerover/blob/main/evaluation_questions.json): `evaluation_questions.json`
- [Question evaluation](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py): `rag_evaluation.py`
- [Semantic search](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py): `rag_evaluation.py`
- [Grounding check](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py): `rag_evaluation.py`
- [LLM call](https://github.com/trailblazer09/referencerover/blob/main/rag_evaluation.py): `rag_evaluation.py`
- **Answers and metrics**

### External services

- **Qdrant**
- **Gemini or Groq**

### Other components

- **User**
- [CLI entry](https://github.com/trailblazer09/referencerover/blob/main/main.py): `main.py`

## Connections

- User → CLI entry: runs
- CLI entry → CLI workflow: invokes
- CLI workflow → PDF extraction: loads PDFs
- Source PDFs → PDF extraction: provides files
- PDF extraction → Text chunking: extracts text
- CLI workflow → Document indexing: indexes documents
- Document indexing → Embeddings: embeds chunks
- Document indexing → Qdrant: writes vectors
- CLI workflow → Question evaluation: evaluates questions
- Evaluation questions → CLI workflow: supplies questions
- Question evaluation → Semantic search: retrieves context
- Semantic search → Embeddings: embeds query
- Semantic search → Qdrant: searches vectors
- Question evaluation → LLM call: requests answer
- LLM call → Gemini or Groq: sends prompt
- Question evaluation → Grounding check: checks context
- Question evaluation → Answers and metrics: returns answer and metrics
- CLI workflow → Answers and metrics: prints references