How haisonle001/queryroute works
QueryRoute is a benchmark and selector toolkit for choosing among fixed query-reformulation candidates. Its main workflow is frozen per-query artifacts → selector choice → evaluation against recorded retrieval outcomes. The repository also includes QPP and RouteLLM routing, cross-ranker training code, and a separate Pyserini evaluation utility. The graph follows README-documented artifact flow and visible source references; relationships among artifact subdirectories and unsampled implementations are not inferred.
Main entry points: routellm_infer_evaluate.py RouteLLM evaluator · evaluate_local.py Local evaluator · dataset Frozen outcomes · qpp_router.py QPP router
Components 11
Benchmark artifacts
- Frozen outcomes
dataset
Candidate selection
- RouteLLM evaluator
src/selectors/routellm_infer_evaluate.py - Batch runner
src/selectors/batch_routellm_infer.py - QPP router
src/selectors/qpp_router.py
Evaluation and scoring
- Local evaluator
src/evaluation/evaluate_local.py - Pyserini evaluator
src/evaluation/evaluate_pyserini.py - Evaluation results
Cross-ranker training
- Ranker datasets
src/selectors/cross_ranker/dataset.py - LLM ranker trainer
src/selectors/cross_ranker/llm_ranker.py - Pairwise ranker trainer
src/selectors/cross_ranker/cross_ranker.py
Other components
- Benchmark user
Connections 15
- Benchmark user → RouteLLM evaluator: runs evaluation
README.md - RouteLLM evaluator → Frozen outcomes: reads artifacts
README.md - RouteLLM evaluator → Evaluation results: emits scores
README.md - Benchmark user → QPP router: runs routing
README.md - QPP router → Frozen outcomes: uses frozen artifacts
README.md - Batch runner → RouteLLM evaluator: launches evaluator
src/selectors/batch_routellm_infer.py - RouteLLM evaluator → Local evaluator: imports
src/selectors/routellm_infer_evaluate.py - Benchmark user → Ranker datasets: provides training data
- LLM ranker trainer → Ranker datasets: uses dataset
src/selectors/cross_ranker/llm_ranker.py - Benchmark user → Pairwise ranker trainer: provides training data
- Benchmark user → Local evaluator: evaluates selections
README.md - Local evaluator → Frozen outcomes: scores against outcomes
README.md - Local evaluator → Evaluation results: reports metrics
- Benchmark user → Pyserini evaluator: evaluates retrieval run
src/evaluation/evaluate_pyserini.py - Pyserini evaluator → Evaluation results: writes metrics
src/evaluation/evaluate_pyserini.py