# autonoma-tools/how-to-test-rag-retrieval architecture

> Architecture diagram of the GitHub repository autonoma-tools/how-to-test-rag-retrieval, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/autonoma-tools/how-to-test-rag-retrieval
- Repository: https://github.com/autonoma-tools/how-to-test-rag-retrieval

## Overview

This repository is an offline harness for evaluating RAG retrieval without generation, network access, or a vector database. The main workflow embeds a query, returns ranked chunk IDs, and scores those results against labeled query-to-chunk pairs using hit rate, MRR, context precision, and context recall. The default corpus and deterministic embedder are lightweight internal layers supporting that workflow. The README documents the runnable report and regression gate; tests and CI machinery are omitted from this runtime map.

Main entry points: [`examples/run_retrieval.py`](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/examples/run_retrieval.py) (Retrieval report demo), [`src/metrics/mrr.py`](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/mrr.py) (Mean reciprocal rank), [`src/metrics/hit_rate.py`](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/hit_rate.py) (Hit rate), [`src/retriever.py`](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/retriever.py) (Retriever)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_retrieval["Retrieval"]
  node_retriever["Retriever<br/>[retriever.py]"]
  node_corpus["Help corpus<br/>[corpus.py]"]
  node_embedder["Hashing embedder<br/>[embedding.py]"]
  node_ranked_ids["Ranked chunk IDs"]
end

subgraph group_evaluation["Evaluation"]
  node_hit_rate["Hit rate<br/>[hit_rate.py]"]
  node_mrr["Mean reciprocal rank<br/>[mrr.py]"]
  node_context_metrics["Context precision and recall"]
  node_metric_guards["Metric input guards<br/>[common.py]"]
end

subgraph group_dataset["Evaluation data"]
  node_labels["Labeled query pairs"]
end

subgraph group_reporting["Reporting"]
  node_demo["Retrieval report demo<br/>[run_retrieval.py]"]
  node_aggregate_results["Aggregate metric results"]
end

node_user(("Engineer"))

node_user -->|"runs"| node_demo
node_demo -->|"queries"| node_retriever
node_retriever -->|"loads chunks"| node_corpus
node_retriever -->|"embeds text"| node_embedder
node_retriever -->|"returns"| node_ranked_ids
node_demo -->|"scores against"| node_labels
node_demo -->|"computes"| node_hit_rate
node_demo -->|"computes"| node_mrr
node_demo -->|"computes"| node_context_metrics
node_ranked_ids -->|"scores"| node_hit_rate
node_ranked_ids -->|"scores"| node_mrr
node_ranked_ids -->|"scores"| node_context_metrics
node_hit_rate -->|"uses"| node_metric_guards
node_mrr -->|"uses"| node_metric_guards
node_context_metrics -->|"uses"| node_metric_guards
node_mrr -->|"delegates mean"| node_hit_rate
node_hit_rate -->|"produces scores"| node_aggregate_results
node_mrr -->|"produces scores"| node_aggregate_results
node_context_metrics -->|"produces scores"| node_aggregate_results
node_demo -->|"prints"| node_aggregate_results

click node_retriever "https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/retriever.py"
click node_corpus "https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/corpus.py"
click node_embedder "https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/embedding.py"
click node_labels "https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/tests/fixtures/labeled_queries.json"
click node_hit_rate "https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/hit_rate.py"
click node_mrr "https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/mrr.py"
click node_context_metrics "https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/context_precision_recall.py"
click node_metric_guards "https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/common.py"
click node_demo "https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/examples/run_retrieval.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_retriever,node_corpus,node_embedder,node_ranked_ids toneBlue
class node_hit_rate,node_mrr,node_context_metrics,node_metric_guards toneAmber
class node_labels toneMint
class node_demo,node_aggregate_results toneRose
class node_user toneIndigo
```

## Components

### Retrieval

- [Retriever](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/retriever.py): `src/retriever.py`
- [Help corpus](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/corpus.py): `src/corpus.py`
- [Hashing embedder](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/embedding.py): `src/embedding.py`
- **Ranked chunk IDs**

### Evaluation

- [Hit rate](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/hit_rate.py): `src/metrics/hit_rate.py`
- [Mean reciprocal rank](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/mrr.py): `src/metrics/mrr.py`
- [Context precision and recall](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/context_precision_recall.py): `src/metrics/context_precision_recall.py`
- [Metric input guards](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/src/metrics/common.py): `src/metrics/common.py`

### Evaluation data

- [Labeled query pairs](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/tests/fixtures/labeled_queries.json): `tests/fixtures/labeled_queries.json`

### Reporting

- [Retrieval report demo](https://github.com/autonoma-tools/how-to-test-rag-retrieval/blob/main/examples/run_retrieval.py): `examples/run_retrieval.py`
- **Aggregate metric results**

### Other components

- **Engineer**

## Connections

- Engineer → Retrieval report demo: runs (evidence: `README.md`)
- Retrieval report demo → Retriever: queries (evidence: `README.md`)
- Retriever → Help corpus: loads chunks (evidence: `src/retriever.py`)
- Retriever → Hashing embedder: embeds text (evidence: `src/retriever.py`)
- Retriever → Ranked chunk IDs: returns (evidence: `src/retriever.py`)
- Retrieval report demo → Labeled query pairs: scores against (evidence: `README.md`)
- Retrieval report demo → Hit rate: computes (evidence: `README.md`)
- Retrieval report demo → Mean reciprocal rank: computes (evidence: `README.md`)
- Retrieval report demo → Context precision and recall: computes (evidence: `README.md`)
- Ranked chunk IDs → Hit rate: scores (evidence: `README.md`)
- Ranked chunk IDs → Mean reciprocal rank: scores (evidence: `README.md`)
- Ranked chunk IDs → Context precision and recall: scores (evidence: `README.md`)
- Hit rate → Metric input guards: uses (evidence: `src/metrics/hit_rate.py`)
- Mean reciprocal rank → Metric input guards: uses (evidence: `src/metrics/mrr.py`)
- Context precision and recall → Metric input guards: uses (evidence: `src/metrics/context_precision_recall.py`)
- Mean reciprocal rank → Hit rate: delegates mean (evidence: `src/metrics/mrr.py`)
- Hit rate → Aggregate metric results: produces scores (evidence: `README.md`)
- Mean reciprocal rank → Aggregate metric results: produces scores (evidence: `README.md`)
- Context precision and recall → Aggregate metric results: produces scores (evidence: `README.md`)
- Retrieval report demo → Aggregate metric results: prints (evidence: `README.md`)
