# xtraluck21/r2r-router-optimization architecture

> Architecture diagram of the GitHub repository xtraluck21/r2r-router-optimization, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xtraluck21/r2r-router-optimization
- Repository: https://github.com/xtraluck21/r2r-router-optimization

## Overview

This repository is an experimental pipeline for learning when a Qwen multiple-choice model is likely to be wrong, then selectively using retrieval-augmented generation (RAG). The main workflow runs model inference, turns correctness and uncertainty signals into router features, trains a failure predictor, and compares vanilla, always-retrieve, and router-triggered answers. Wikipedia retrieval and similarity filtering form a distinct optional branch. The README documents the workflow; unsampled R2R internals and notebook experiments are not expanded beyond documented behavior.

Main entry points: [`src/r2r_baseline_comparison.py`](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/r2r_baseline_comparison.py) (R2R comparison), [`src/inference_qwen_mcq.py`](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/inference_qwen_mcq.py) (Qwen inference), [`src/trainer.py`](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/trainer.py) (Router training), [`src/prepare_dataset.py`](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/prepare_dataset.py) (Feature matrix)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Inference"]
  node_inference_run["Qwen inference"]
  node_mps_check["MPS check<br/>[mps_gpu_check.py]"]
  node_inference_results["Inference results"]
end

subgraph group_router["Router training"]
  node_feature_matrix["Feature matrix<br/>[prepare_dataset.py]"]
  node_feature_builder["Feature extraction<br/>[prepare_dataset.py]"]
  node_router_model["Trained router<br/>[trainer.py]"]
  node_router_trainer["Router training<br/>[trainer.py]"]
  node_router_outputs["Router artifacts<br/>[trainer.py]"]
end

subgraph group_retrieval["RAG evaluation"]
  node_r2r["R2R comparison"]
  node_rag["Wikipedia retrieval<br/>[rag_pipeline.py]"]
  node_similarity["Passage filtering<br/>[rag_pipeline.py]"]
  node_evaluation_results["R2R results"]
end

subgraph group_reporting["Reporting"]
  node_report_visuals["Report figures"]
  node_static_charts["Static charts<br/>[report_charts.py]"]
end

node_researcher(("Researcher"))
node_questions["MCQ splits<br/>[Train.csv]"]
node_huggingface["Hugging Face model"]
node_wikipedia["Wikipedia"]
node_embed_model["Sentence-BERT"]

node_researcher -->|"runs"| node_inference_run
node_questions -->|"loads"| node_inference_run
node_inference_run -->|"loads model"| node_huggingface
node_inference_run -->|"checks device"| node_mps_check
node_inference_run -->|"writes"| node_inference_results
node_inference_results -->|"loads"| node_feature_builder
node_feature_builder -->|"builds"| node_feature_matrix
node_feature_matrix -->|"loads"| node_router_trainer
node_router_trainer -->|"saves"| node_router_model
node_router_trainer -->|"writes reports"| node_router_outputs
node_researcher -->|"runs evaluation"| node_r2r
node_router_model -->|"supplies predictions"| node_r2r
node_feature_matrix -->|"supplies features"| node_r2r
node_questions -->|"supplies test set"| node_r2r
node_r2r -->|"retrieves context"| node_rag
node_rag -->|"searches"| node_wikipedia
node_rag -->|"filters passages"| node_similarity
node_similarity -->|"encodes text"| node_embed_model
node_r2r -->|"writes comparison"| node_evaluation_results
node_researcher -->|"generates figures"| node_report_visuals
node_router_trainer -->|"shares training logic"| node_report_visuals
node_researcher -->|"generates charts"| node_static_charts

click node_questions "https://github.com/xtraluck21/r2r-router-optimization/blob/main/data/raw/Train.csv"
click node_inference_run "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/inference_qwen_mcq.py"
click node_mps_check "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/mps_gpu_check.py"
click node_inference_results "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/inference_qwen_mcq.py"
click node_feature_matrix "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/prepare_dataset.py"
click node_feature_builder "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/prepare_dataset.py"
click node_router_model "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/trainer.py"
click node_router_trainer "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/trainer.py"
click node_r2r "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/r2r_baseline_comparison.py"
click node_rag "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/rag_pipeline.py"
click node_similarity "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/rag_pipeline.py"
click node_evaluation_results "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/r2r_baseline_comparison.py"
click node_report_visuals "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/generate_report_visuals.py"
click node_static_charts "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/report_charts.py"
click node_router_outputs "https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/trainer.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_inference_run,node_mps_check,node_inference_results toneBlue
class node_feature_matrix,node_feature_builder,node_router_model,node_router_trainer,node_router_outputs toneAmber
class node_r2r,node_rag,node_similarity,node_evaluation_results toneMint
class node_report_visuals,node_static_charts toneRose
class node_researcher,node_huggingface,node_wikipedia,node_embed_model toneIndigo
class node_questions toneTeal
```

## Components

### Inference

- [Qwen inference](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/inference_qwen_mcq.py): `src/inference_qwen_mcq.py`
- [MPS check](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/mps_gpu_check.py): `src/mps_gpu_check.py`
- [Inference results](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/inference_qwen_mcq.py): `src/inference_qwen_mcq.py`

### Router training

- [Feature matrix](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/prepare_dataset.py): `src/prepare_dataset.py`
- [Feature extraction](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/prepare_dataset.py): `src/prepare_dataset.py`
- [Trained router](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/trainer.py): `src/trainer.py`
- [Router training](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/trainer.py): `src/trainer.py`
- [Router artifacts](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/trainer.py): `src/trainer.py`

### RAG evaluation

- [R2R comparison](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/r2r_baseline_comparison.py): `src/r2r_baseline_comparison.py`
- [Wikipedia retrieval](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/rag_pipeline.py): `src/rag_pipeline.py`
- [Passage filtering](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/rag_pipeline.py): `src/rag_pipeline.py`
- [R2R results](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/r2r_baseline_comparison.py): `src/r2r_baseline_comparison.py`

### Reporting

- [Report figures](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/generate_report_visuals.py): `src/generate_report_visuals.py`
- [Static charts](https://github.com/xtraluck21/r2r-router-optimization/blob/main/src/report_charts.py): `src/report_charts.py`

### Other components

- **Researcher**
- [MCQ splits](https://github.com/xtraluck21/r2r-router-optimization/blob/main/data/raw/Train.csv): `data/raw/Train.csv`
- **Hugging Face model**
- **Wikipedia**
- **Sentence-BERT**

## Connections

- Researcher → Qwen inference: runs (evidence: `README.md`)
- MCQ splits → Qwen inference: loads (evidence: `README.md`)
- Qwen inference → Hugging Face model: loads model (evidence: `README.md`)
- Qwen inference → MPS check: checks device (evidence: `src/inference_qwen_mcq.py`)
- Qwen inference → Inference results: writes (evidence: `README.md`)
- Inference results → Feature extraction: loads (evidence: `README.md`)
- Feature extraction → Feature matrix: builds (evidence: `README.md`)
- Feature matrix → Router training: loads (evidence: `README.md`)
- Router training → Trained router: saves (evidence: `README.md`)
- Router training → Router artifacts: writes reports (evidence: `README.md`)
- Researcher → R2R comparison: runs evaluation (evidence: `README.md`)
- Trained router → R2R comparison: supplies predictions (evidence: `README.md`)
- Feature matrix → R2R comparison: supplies features (evidence: `README.md`)
- MCQ splits → R2R comparison: supplies test set (evidence: `README.md`)
- R2R comparison → Wikipedia retrieval: retrieves context (evidence: `src/r2r_baseline_comparison.py`)
- Wikipedia retrieval → Wikipedia: searches (evidence: `README.md`)
- Wikipedia retrieval → Passage filtering: filters passages (evidence: `src/rag_pipeline.py`)
- Passage filtering → Sentence-BERT: encodes text (evidence: `README.md`)
- R2R comparison → R2R results: writes comparison (evidence: `README.md`)
- Researcher → Report figures: generates figures (evidence: `README.md`)
- Router training → Report figures: shares training logic (evidence: `src/generate_report_visuals.py`)
- Researcher → Static charts: generates charts (evidence: `README.md`)
