# haisonle001/queryroute architecture

> Architecture diagram of the GitHub repository haisonle001/queryroute, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/haisonle001/queryroute
- Repository: https://github.com/haisonle001/queryroute

## Overview

QueryRoute is a benchmark and selector toolkit for choosing among fixed query-reformulation candidates. Its main workflow is frozen per-query artifacts → selector choice → evaluation against recorded retrieval outcomes. The repository also includes QPP and RouteLLM routing, cross-ranker training code, and a separate Pyserini evaluation utility. The graph follows README-documented artifact flow and visible source references; relationships among artifact subdirectories and unsampled implementations are not inferred.

Main entry points: [`src/selectors/routellm_infer_evaluate.py`](https://github.com/haisonle001/queryroute/blob/main/src/selectors/routellm_infer_evaluate.py) (RouteLLM evaluator), [`src/evaluation/evaluate_local.py`](https://github.com/haisonle001/queryroute/blob/main/src/evaluation/evaluate_local.py) (Local evaluator), [`dataset`](https://github.com/haisonle001/queryroute/tree/main/dataset) (Frozen outcomes), [`src/selectors/qpp_router.py`](https://github.com/haisonle001/queryroute/blob/main/src/selectors/qpp_router.py) (QPP router)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_artifacts["Benchmark artifacts"]
  node_frozen[("Frozen outcomes")]
end

subgraph group_selection["Candidate selection"]
  node_route["RouteLLM evaluator"]
  node_batch["Batch runner"]
  node_qpp["QPP router<br/>[qpp_router.py]"]
end

subgraph group_scoring["Evaluation and scoring"]
  node_localeval["Local evaluator<br/>[evaluate_local.py]"]
  node_pyserini["Pyserini evaluator"]
  node_results["Evaluation results"]
end

subgraph group_ranking["Cross-ranker training"]
  node_crossdata["Ranker datasets<br/>[dataset.py]"]
  node_llmrank["LLM ranker trainer<br/>[llm_ranker.py]"]
  node_crossrank["Pairwise ranker trainer<br/>[cross_ranker.py]"]
end

node_user(("Benchmark user"))

node_user -->|"runs evaluation"| node_route
node_route -->|"reads artifacts"| node_frozen
node_route -->|"emits scores"| node_results
node_user -->|"runs routing"| node_qpp
node_qpp -->|"uses frozen artifacts"| node_frozen
node_batch -->|"launches evaluator"| node_route
node_route -->|"imports"| node_localeval
node_user -.->|"provides training data"| node_crossdata
node_llmrank -->|"uses dataset"| node_crossdata
node_user -.->|"provides training data"| node_crossrank
node_user -->|"evaluates selections"| node_localeval
node_localeval -->|"scores against outcomes"| node_frozen
node_localeval -.->|"reports metrics"| node_results
node_user -->|"evaluates retrieval run"| node_pyserini
node_pyserini -->|"writes metrics"| node_results

click node_frozen "https://github.com/haisonle001/queryroute/tree/main/dataset"
click node_route "https://github.com/haisonle001/queryroute/blob/main/src/selectors/routellm_infer_evaluate.py"
click node_batch "https://github.com/haisonle001/queryroute/blob/main/src/selectors/batch_routellm_infer.py"
click node_qpp "https://github.com/haisonle001/queryroute/blob/main/src/selectors/qpp_router.py"
click node_crossdata "https://github.com/haisonle001/queryroute/blob/main/src/selectors/cross_ranker/dataset.py"
click node_llmrank "https://github.com/haisonle001/queryroute/blob/main/src/selectors/cross_ranker/llm_ranker.py"
click node_crossrank "https://github.com/haisonle001/queryroute/blob/main/src/selectors/cross_ranker/cross_ranker.py"
click node_localeval "https://github.com/haisonle001/queryroute/blob/main/src/evaluation/evaluate_local.py"
click node_pyserini "https://github.com/haisonle001/queryroute/blob/main/src/evaluation/evaluate_pyserini.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frozen,node_user toneBlue
class node_route,node_batch,node_qpp toneAmber
class node_localeval,node_pyserini,node_results toneMint
class node_crossdata,node_llmrank,node_crossrank toneRose
```

## Components

### Benchmark artifacts

- [Frozen outcomes](https://github.com/haisonle001/queryroute/tree/main/dataset): `dataset`

### Candidate selection

- [RouteLLM evaluator](https://github.com/haisonle001/queryroute/blob/main/src/selectors/routellm_infer_evaluate.py): `src/selectors/routellm_infer_evaluate.py`
- [Batch runner](https://github.com/haisonle001/queryroute/blob/main/src/selectors/batch_routellm_infer.py): `src/selectors/batch_routellm_infer.py`
- [QPP router](https://github.com/haisonle001/queryroute/blob/main/src/selectors/qpp_router.py): `src/selectors/qpp_router.py`

### Evaluation and scoring

- [Local evaluator](https://github.com/haisonle001/queryroute/blob/main/src/evaluation/evaluate_local.py): `src/evaluation/evaluate_local.py`
- [Pyserini evaluator](https://github.com/haisonle001/queryroute/blob/main/src/evaluation/evaluate_pyserini.py): `src/evaluation/evaluate_pyserini.py`
- **Evaluation results**

### Cross-ranker training

- [Ranker datasets](https://github.com/haisonle001/queryroute/blob/main/src/selectors/cross_ranker/dataset.py): `src/selectors/cross_ranker/dataset.py`
- [LLM ranker trainer](https://github.com/haisonle001/queryroute/blob/main/src/selectors/cross_ranker/llm_ranker.py): `src/selectors/cross_ranker/llm_ranker.py`
- [Pairwise ranker trainer](https://github.com/haisonle001/queryroute/blob/main/src/selectors/cross_ranker/cross_ranker.py): `src/selectors/cross_ranker/cross_ranker.py`

### Other components

- **Benchmark user**

## Connections

- Benchmark user → RouteLLM evaluator: runs evaluation (evidence: `README.md`)
- RouteLLM evaluator → Frozen outcomes: reads artifacts (evidence: `README.md`)
- RouteLLM evaluator → Evaluation results: emits scores (evidence: `README.md`)
- Benchmark user → QPP router: runs routing (evidence: `README.md`)
- QPP router → Frozen outcomes: uses frozen artifacts (evidence: `README.md`)
- Batch runner → RouteLLM evaluator: launches evaluator (evidence: `src/selectors/batch_routellm_infer.py`)
- RouteLLM evaluator → Local evaluator: imports (evidence: `src/selectors/routellm_infer_evaluate.py`)
- Benchmark user → Ranker datasets: provides training data
- LLM ranker trainer → Ranker datasets: uses dataset (evidence: `src/selectors/cross_ranker/llm_ranker.py`)
- Benchmark user → Pairwise ranker trainer: provides training data
- Benchmark user → Local evaluator: evaluates selections (evidence: `README.md`)
- Local evaluator → Frozen outcomes: scores against outcomes (evidence: `README.md`)
- Local evaluator → Evaluation results: reports metrics
- Benchmark user → Pyserini evaluator: evaluates retrieval run (evidence: `src/evaluation/evaluate_pyserini.py`)
- Pyserini evaluator → Evaluation results: writes metrics (evidence: `src/evaluation/evaluate_pyserini.py`)
