How xtraluck21/r2r-router-optimization works
This repository is an experimental pipeline for learning when a Qwen multiple-choice model is likely to be wrong, then selectively using retrieval-augmented generation (RAG). The main workflow runs model inference, turns correctness and uncertainty signals into router features, trains a failure predictor, and compares vanilla, always-retrieve, and router-triggered answers. Wikipedia retrieval and similarity filtering form a distinct optional branch. The README documents the workflow; unsampled R2R internals and notebook experiments are not expanded beyond documented behavior.
Main entry points: r2r_baseline_comparison.py R2R comparison · inference_qwen_mcq.py Qwen inference · trainer.py Router training · prepare_dataset.py Feature matrix
Components 19
Inference
- Qwen inference
src/inference_qwen_mcq.py - MPS check
src/mps_gpu_check.py - Inference results
src/inference_qwen_mcq.py
Router training
- Feature matrix
src/prepare_dataset.py - Feature extraction
src/prepare_dataset.py - Trained router
src/trainer.py - Router training
src/trainer.py - Router artifacts
src/trainer.py
RAG evaluation
- R2R comparison
src/r2r_baseline_comparison.py - Wikipedia retrieval
src/rag_pipeline.py - Passage filtering
src/rag_pipeline.py - R2R results
src/r2r_baseline_comparison.py
Reporting
- Report figures
src/generate_report_visuals.py - Static charts
src/report_charts.py
Other components
- Researcher
- MCQ splits
data/raw/Train.csv - Hugging Face model
- Wikipedia
- Sentence-BERT
Connections 22
- Researcher → Qwen inference: runs
README.md - MCQ splits → Qwen inference: loads
README.md - Qwen inference → Hugging Face model: loads model
README.md - Qwen inference → MPS check: checks device
src/inference_qwen_mcq.py - Qwen inference → Inference results: writes
README.md - Inference results → Feature extraction: loads
README.md - Feature extraction → Feature matrix: builds
README.md - Feature matrix → Router training: loads
README.md - Router training → Trained router: saves
README.md - Router training → Router artifacts: writes reports
README.md - Researcher → R2R comparison: runs evaluation
README.md - Trained router → R2R comparison: supplies predictions
README.md - Feature matrix → R2R comparison: supplies features
README.md - MCQ splits → R2R comparison: supplies test set
README.md - R2R comparison → Wikipedia retrieval: retrieves context
src/r2r_baseline_comparison.py - Wikipedia retrieval → Wikipedia: searches
README.md - Wikipedia retrieval → Passage filtering: filters passages
src/rag_pipeline.py - Passage filtering → Sentence-BERT: encodes text
README.md - R2R comparison → R2R results: writes comparison
README.md - Researcher → Report figures: generates figures
README.md - Router training → Report figures: shares training logic
src/generate_report_visuals.py - Researcher → Static charts: generates charts
README.md