How bhatraasim/qwen-mathematical-reasoning-model works
This repository implements an experimental GSM8K math-reasoning pipeline: generate verified teacher traces, prepare tagged data, train an SFT adapter, continue with GRPO rewards, evaluate models, and analyze saved runs. The README documents the main stages; source excerpts show the core data preparation, training, reward, and evaluation behavior. Some evaluation and analysis details are only partially excerpted, so those links rely on documented workflow or are omitted where wiring is uncertain.
Main entry points: train_grpo.py GRPO training · evaluate_all.py Model evaluation · summarize_runs.py Run summaries · generate_data.py Trace generation
Components 16
Data preparation
- Trace generation
data_generation/generate_data.py - Tagged SFT data
data_generation/prepare_tagged_dataset.py - GRPO prompts
data_generation/prepare_grpo_dataset.py
Model training
- SFT training
scripts/train_sft.py - GRPO rewards
scripts/grpo_rewards.py - GRPO training
scripts/train_grpo.py - Saved adapters
scripts/train_grpo.py
Evaluation
- Model evaluation
scripts/evaluate_all.py - Evaluation files
scripts/evaluate_all.py
Run analysis
- Run summaries
scripts/summarize_runs.py - Paired comparison
scripts/compare_runs.py - Summary reports
scripts/summarize_runs.py
Other components
- Experimenter
- GSM8K
- Teacher model
- Qwen base model
Connections 25
- Experimenter → Trace generation: starts
README.md - GSM8K → Trace generation: supplies questions
data_generation/generate_data.py - Teacher model → Trace generation: generates traces
data_generation/generate_data.py - Trace generation → Tagged SFT data: writes traces
data_generation/prepare_tagged_dataset.py - Experimenter → SFT training: starts
README.md - Tagged SFT data → SFT training: provides training data
scripts/train_sft.py - Qwen base model → SFT training: initializes model
scripts/train_sft.py - Experimenter → GRPO training: starts
README.md - GSM8K → GRPO prompts: supplies prompts
data_generation/prepare_grpo_dataset.py - Tagged SFT data → GRPO prompts: shares prompt format
data_generation/prepare_grpo_dataset.py - GRPO prompts → GRPO training: provides training data
scripts/train_grpo.py - Saved adapters → GRPO training: initializes policy
scripts/train_grpo.py - GRPO rewards → GRPO training: scores completions
scripts/train_grpo.py - SFT training → Saved adapters: saves adapter
scripts/train_sft.py - GRPO training → Saved adapters: saves adapter
scripts/train_grpo.py - Experimenter → Model evaluation: runs evaluation
README.md - GSM8K → Model evaluation: provides test split
scripts/evaluate_all.py - Qwen base model → Model evaluation: provides model
- Saved adapters → Model evaluation: provides adapter
README.md - Model evaluation → Evaluation files: writes results
scripts/evaluate_all.py - Experimenter → Run summaries: starts analysis
README.md - Evaluation files → Run summaries: reads runs
scripts/summarize_runs.py - Run summaries → Paired comparison: uses comparison logic
scripts/summarize_runs.py - Run summaries → Summary reports: writes reports
scripts/summarize_runs.py - Experimenter → Paired comparison: compares runs
scripts/compare_runs.py