How dch2r/chasing-the-aha-moment works
This repository is an instrumented GRPO post-training and evaluation study, not a new RL algorithm. Its main workflow trains a reward-design arm, records per-response rewards and training diagnostics, evaluates the saved adapter against the same base model under a fixed protocol, then compares paired outcomes and exports changed responses. It also supports resumable GPU replay and CPU-side evidence verification. The graph includes the documented ablation and replay branches; unsampled notebook wiring is represented only where the README establishes it.
Main entry points: train_arm.py Train reward arm · evaluate.py Paired evaluator · experiment_rewards.py Reward arms · eval_scoring.py Evaluation scoring
Components 25
Training and rewards
- Train reward arm
scripts/train_arm.py - Reward arms
src/experiment_rewards.py - Training parser
src/rewards.py - Response logger
src/arm_logging.py - Training diagnostics
src/diagnostics.py - GRPO compatibility
src/grpo_compat.py
Evaluation
- Paired evaluator
scripts/evaluate.py - Evaluation scoring
src/eval_scoring.py - Resumable evaluation I/O
src/evaluation_io.py - Adapter fingerprints
src/artifact_checks.py
Analysis and audit
- Result analyzer
scripts/analyze_results.py - Paired statistics
src/paired_analysis.py - Ablation summarizer
scripts/summarize_ablation.py - Evidence verifier
scripts/verify_results.py - Saved-answer demo
scripts/demo.py
Saved evidence
- Saved LoRA adapter
adapter - Pilot training records
results/pilot/run_metadata - Training logs
results/pilot - Evaluation results
results/evaluation - Replay results
results/replay - Changed responses
results/analysis/changed_responses.json - Ablation logs
results/ablation/stage1
External dependencies
- TRL GRPO
- Hugging Face model
Other components
- Researcher
Connections 35
- Researcher → Train reward arm: selects arm
README.md - Train reward arm → Pilot training records: loads records
scripts/train_arm.py - Train reward arm → Hugging Face model: loads model
scripts/train_arm.py - Train reward arm → Reward arms: selects reward
scripts/train_arm.py - Reward arms → Training parser: scores historically
src/experiment_rewards.py - Reward arms → Evaluation scoring: scores corrected
src/experiment_rewards.py - Train reward arm → Response logger: creates logger
scripts/train_arm.py - Response logger → Reward arms: computes components
src/arm_logging.py - Response logger → Training logs: writes responses
src/arm_logging.py - Train reward arm → GRPO compatibility: configures training
scripts/train_arm.py - Train reward arm → Training diagnostics: summarizes run
scripts/train_arm.py - Train reward arm → TRL GRPO: trains with
scripts/train_arm.py - Train reward arm → Saved LoRA adapter: saves adapter
scripts/train_arm.py - Researcher → Paired evaluator: runs evaluation
README.md - Paired evaluator → Hugging Face model: loads base
README.md - Paired evaluator → Saved LoRA adapter: loads adapter
README.md - Paired evaluator → Adapter fingerprints: checks fingerprints
scripts/evaluate.py - Paired evaluator → Evaluation scoring: scores answers
scripts/evaluate.py - Paired evaluator → Resumable evaluation I/O: resumes and writes
scripts/evaluate.py - Paired evaluator → Evaluation results: records evaluation
README.md - Paired evaluator → Replay results: records replay
README.md - Researcher → Result analyzer: analyzes results
README.md - Result analyzer → Evaluation scoring: scores responses
scripts/analyze_results.py - Result analyzer → Paired statistics: computes paired tests
scripts/analyze_results.py - Result analyzer → Changed responses: exports changes
README.md - Researcher → Ablation summarizer: summarizes ablation
README.md - Ablation summarizer → Reward arms: uses arm scoring
scripts/summarize_ablation.py - Ablation summarizer → Ablation logs: summarizes runs
README.md - Researcher → Evidence verifier: audits evidence
README.md - Evidence verifier → Evaluation scoring: rechecks scores
scripts/verify_results.py - Evidence verifier → Adapter fingerprints: checks adapter
scripts/verify_results.py - Evidence verifier → Paired statistics: recomputes analysis
scripts/verify_results.py - Researcher → Saved-answer demo: replays saved answers
README.md - Saved-answer demo → Evaluation scoring: scores live
scripts/demo.py - Saved-answer demo → Training logs: reads saved run
README.md