# wangbo-7423/finetune architecture

> Architecture diagram of the GitHub repository wangbo-7423/finetune, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wangbo-7423/finetune
- Repository: https://github.com/wangbo-7423/finetune

## Overview

This repository documents and implements GSM8K fine-tuning and evaluation for Qwen3.5-4B. The main workflow prepares datasets, trains a LoRA SFT adapter, applies rule-reward GRPO, merges both adapters back into a complete model, and evaluates results; chat and visualization support inspection. Preference-pair data is prepared for a reward-model stage, but the README says GRPO uses rule rewards and no reward-model trainer is present. Edges reflect shown code or README statements; the source excerpts do not establish additional runtime wiring.

Main entry points: [`qwen-finetune/train_grpo.py`](https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/train_grpo.py) (Rule-reward GRPO), [`qwen-finetune/merge_grpo.py`](https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/merge_grpo.py) (Adapter merge), [`prepare_gsm8k.py`](https://github.com/wangbo-7423/finetune/blob/main/prepare_gsm8k.py) (Dataset preparation), [`qwen-finetune/strain_sft.py`](https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/strain_sft.py) (SFT training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["GSM8K data"]
  node_gsm8k[("Source GSM8K")]
  node_prep["Dataset preparation<br/>[prepare_gsm8k.py]"]
  node_stage_data[("Prepared splits")]
  node_preference_data["Preference pairs<br/>[rm_train.jsonl]"]
end

subgraph group_training["Model training"]
  node_sft["SFT training<br/>[strain_sft.py]"]
  node_grpo["Rule-reward GRPO<br/>[train_grpo.py]"]
end

subgraph group_inference["Inference and evaluation"]
  node_eval["GSM8K evaluation<br/>[eval_gsm8k.py]"]
  node_eval_results["Evaluation results"]
  node_chat["Interactive chat<br/>[chat.py]"]
  node_visualize["Result visualization<br/>[visualize.py]"]
end

subgraph group_artifacts["Model artifacts"]
  node_base_model[("Qwen base model")]
  node_sft_adapter[("SFT adapter")]
  node_grpo_adapter[("GRPO adapter")]
  node_merge["Adapter merge<br/>[merge_grpo.py]"]
  node_merged_model[("Merged model")]
end

node_engineer(("Engineer"))

node_engineer -->|"runs preparation"| node_prep
node_prep -->|"reads parquet"| node_gsm8k
node_prep -->|"writes splits"| node_stage_data
node_prep -->|"builds pairs"| node_preference_data
node_engineer -->|"starts training"| node_sft
node_sft -->|"loads base"| node_base_model
node_sft -->|"loads SFT splits"| node_stage_data
node_sft -->|"saves adapter"| node_sft_adapter
node_engineer -->|"starts training"| node_grpo
node_grpo -->|"loads base"| node_base_model
node_grpo -->|"merges SFT"| node_sft_adapter
node_grpo -->|"loads RL prompts"| node_stage_data
node_grpo -->|"saves adapter"| node_grpo_adapter
node_engineer -->|"runs merge"| node_merge
node_merge -->|"loads base"| node_base_model
node_merge -->|"applies adapter"| node_sft_adapter
node_merge -->|"applies adapter"| node_grpo_adapter
node_merge -->|"saves model"| node_merged_model
node_engineer -->|"runs evaluation"| node_eval
node_eval -->|"reads test set"| node_stage_data
node_eval -->|"loads model"| node_merged_model
node_eval -->|"writes predictions"| node_eval_results
node_engineer -->|"starts chat"| node_chat
node_chat -->|"loads SFT model"| node_sft_adapter
node_visualize -->|"reads results"| node_eval_results

click node_gsm8k "https://github.com/wangbo-7423/finetune/tree/main/datasets/gsm8k"
click node_prep "https://github.com/wangbo-7423/finetune/blob/main/prepare_gsm8k.py"
click node_stage_data "https://github.com/wangbo-7423/finetune/tree/main/data/gsm8k"
click node_preference_data "https://github.com/wangbo-7423/finetune/blob/main/data/gsm8k/rm_train.jsonl"
click node_sft "https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/strain_sft.py"
click node_sft_adapter "https://github.com/wangbo-7423/finetune/tree/main/qwen-finetune/outputs/sft"
click node_grpo "https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/train_grpo.py"
click node_grpo_adapter "https://github.com/wangbo-7423/finetune/tree/main/qwen-finetune/outputs/grpo"
click node_merge "https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/merge_grpo.py"
click node_merged_model "https://github.com/wangbo-7423/finetune/tree/main/qwen-finetune/outputs/grpo_merged"
click node_eval "https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/eval_gsm8k.py"
click node_eval_results "https://github.com/wangbo-7423/finetune/tree/main/qwen-finetune/outputs"
click node_chat "https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/chat.py"
click node_visualize "https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/visualize.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_gsm8k,node_prep,node_stage_data,node_preference_data toneBlue
class node_sft,node_grpo toneAmber
class node_eval,node_eval_results,node_chat,node_visualize toneMint
class node_base_model,node_sft_adapter,node_grpo_adapter,node_merge,node_merged_model toneRose
class node_engineer toneIndigo
```

## Components

### GSM8K data

- [Source GSM8K](https://github.com/wangbo-7423/finetune/tree/main/datasets/gsm8k): `datasets/gsm8k`
- [Dataset preparation](https://github.com/wangbo-7423/finetune/blob/main/prepare_gsm8k.py): `prepare_gsm8k.py`
- [Prepared splits](https://github.com/wangbo-7423/finetune/tree/main/data/gsm8k): `data/gsm8k`
- [Preference pairs](https://github.com/wangbo-7423/finetune/blob/main/data/gsm8k/rm_train.jsonl): `data/gsm8k/rm_train.jsonl`

### Model training

- [SFT training](https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/strain_sft.py): `qwen-finetune/strain_sft.py`
- [Rule-reward GRPO](https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/train_grpo.py): `qwen-finetune/train_grpo.py`

### Inference and evaluation

- [GSM8K evaluation](https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/eval_gsm8k.py): `qwen-finetune/eval_gsm8k.py`
- [Evaluation results](https://github.com/wangbo-7423/finetune/tree/main/qwen-finetune/outputs): `qwen-finetune/outputs`
- [Interactive chat](https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/chat.py): `qwen-finetune/chat.py`
- [Result visualization](https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/visualize.py): `qwen-finetune/visualize.py`

### Model artifacts

- **Qwen base model**
- [SFT adapter](https://github.com/wangbo-7423/finetune/tree/main/qwen-finetune/outputs/sft): `qwen-finetune/outputs/sft`
- [GRPO adapter](https://github.com/wangbo-7423/finetune/tree/main/qwen-finetune/outputs/grpo): `qwen-finetune/outputs/grpo`
- [Adapter merge](https://github.com/wangbo-7423/finetune/blob/main/qwen-finetune/merge_grpo.py): `qwen-finetune/merge_grpo.py`
- [Merged model](https://github.com/wangbo-7423/finetune/tree/main/qwen-finetune/outputs/grpo_merged): `qwen-finetune/outputs/grpo_merged`

### Other components

- **Engineer**

## Connections

- Engineer → Dataset preparation: runs preparation (evidence: `README.md`)
- Dataset preparation → Source GSM8K: reads parquet (evidence: `prepare_gsm8k.py`)
- Dataset preparation → Prepared splits: writes splits (evidence: `prepare_gsm8k.py`)
- Dataset preparation → Preference pairs: builds pairs (evidence: `prepare_gsm8k.py`)
- Engineer → SFT training: starts training (evidence: `README.md`)
- SFT training → Qwen base model: loads base (evidence: `qwen-finetune/strain_sft.py`)
- SFT training → Prepared splits: loads SFT splits (evidence: `qwen-finetune/strain_sft.py`)
- SFT training → SFT adapter: saves adapter (evidence: `qwen-finetune/strain_sft.py`)
- Engineer → Rule-reward GRPO: starts training (evidence: `README.md`)
- Rule-reward GRPO → Qwen base model: loads base (evidence: `qwen-finetune/train_grpo.py`)
- Rule-reward GRPO → SFT adapter: merges SFT (evidence: `qwen-finetune/train_grpo.py`)
- Rule-reward GRPO → Prepared splits: loads RL prompts (evidence: `qwen-finetune/train_grpo.py`)
- Rule-reward GRPO → GRPO adapter: saves adapter (evidence: `qwen-finetune/train_grpo.py`)
- Engineer → Adapter merge: runs merge (evidence: `README.md`)
- Adapter merge → Qwen base model: loads base (evidence: `qwen-finetune/merge_grpo.py`)
- Adapter merge → SFT adapter: applies adapter (evidence: `qwen-finetune/merge_grpo.py`)
- Adapter merge → GRPO adapter: applies adapter (evidence: `qwen-finetune/merge_grpo.py`)
- Adapter merge → Merged model: saves model (evidence: `qwen-finetune/merge_grpo.py`)
- Engineer → GSM8K evaluation: runs evaluation (evidence: `README.md`)
- GSM8K evaluation → Prepared splits: reads test set (evidence: `qwen-finetune/eval_gsm8k.py`)
- GSM8K evaluation → Merged model: loads model (evidence: `README.md`)
- GSM8K evaluation → Evaluation results: writes predictions (evidence: `qwen-finetune/eval_gsm8k.py`)
- Engineer → Interactive chat: starts chat (evidence: `README.md`)
- Interactive chat → SFT adapter: loads SFT model (evidence: `qwen-finetune/chat.py`)
- Result visualization → Evaluation results: reads results (evidence: `qwen-finetune/visualize.py`)