# turningpoint-ai/visualthinker-r1-zero architecture

> Architecture diagram of the GitHub repository turningpoint-ai/visualthinker-r1-zero, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/turningpoint-ai/visualthinker-r1-zero
- Repository: https://github.com/turningpoint-ai/visualthinker-r1-zero

## Overview

1. **Repository purpose**
   - This repo is an end-to-end pipeline for reproducing a multimodal RL result: training a 2B vision-language model to show an “aha moment” on SAT-style visual reasoning, then evaluating it on CVBench.
   - The architecture is organized around three stages: environment/setup, dataset preparation, and model training/evaluation.

2. **Top-level orchestration**
   - `setup.sh` is the entrypoint for installing dependencies and preparing the environment.
   - The root `README.md` acts as the workflow guide, but the actual implementation is split into `src/data`, `src/open-r1-multimodal`, and `src/eval`.

3. **Data preparation subsystem**
   - `src/data/SAT/prepare_dataset.sh` and `process_dataset.py` form the dataset pipeline for the SAT task.
   - This is the ingestion boundary: raw SAT data is transformed into a training-ready format before any model code runs.
   - The repo does not appear to contain the dataset itself, only the preparation logic.

4. **Training framework boundary**
   - `src/open-r1-multimodal` is a self-contained training package, separate from the dataset and evaluation code.
   - It includes its own `setup.py`, `setup.cfg`, `Makefile`, and Python package under `src/open_r1`, indicating this is the main reusable training codebase rather than a one-off script collection.

5. **Model training workflows**
   - Training is exposed through shell entrypoints: `run_grpo.sh`, `run_grpo_SAT.sh`, `run_sft.sh`, and `run_sft_SAT.sh`.
   - These scripts select the training mode and task variant:
     - **GRPO** for reinforcement-learning-based reasoning training
     - **SFT** for supervised fine-tuning comparisons
     - **SAT** versions for the visual reasoning dataset used in the paper
   - `prepare_2B_base.sh` suggests a separate bootstrap step for initializing the 2B base model or its training state.

6. **Core algorithm implementations**
   - In `src/open_r1`, the major training/evaluation logic is split by capability:
     - `sft.py` for supervised fine-tuning
     - `grpo.py` for GRPO training
     - `generate.py` for inference / rollout generation
     - `evaluate.py` for evaluation utilities
   - This indicates a modular design where data sampling, rollout generation, optimization, and evaluation are separated.

7. **Trainer layer**
   - `src/open_r1/trainer/grpo_trainer.py` is the main custom trainer implementation for RL-style optimization.
   - `src/open_r1/trainer/InternVL2.py` suggests model-specific adaptation for a multimodal backbone or chat format.
   - The trainer layer is the boundary between the general training algorithms and the specific multimodal model implementation.

8. **Distributed training infrastructure**
   - `configs/ddp.yaml`, `configs/zero2.yaml`, and `configs/zero3.yaml` indicate support for distributed execution and DeepSpeed-style ZeRO optimization levels.
   - This matters architecturally because training is designed to run on multi-GPU systems and the config choice changes memory partitioning and parallelization behavior.

9. **Evaluation subsystem**
   - `src/eval/evaluate_Qwen2_VL_CVBench.py` and `evaluate_Qwen2_VL_CVBench-base.py` are separate evaluation entrypoints for different model variants.
   - The split between `base` and non-`base` scripts reflects two model families or prompting setups:
     - base model vs. instruct-tuned model
     - with or without reasoning prompt
   - Evaluation is an externalized stage, not embedded in training.

10. **End-to-end data flow**
   - Flow is: raw SAT data → dataset processing → GRPO/SFT training on the open-r1 multimodal package → saved checkpoints → CVBench evaluation scripts.
   - The trained checkpoint is also published externally to Hugging Face, so model artifacts are an output boundary of the repo.

11. **External dependencies and references that shape the architecture**
   - The project is built around existing multimodal and RL foundations: Qwen2-VL, Open-R1, Open-R1-Multimodal, DeepSeek-R1, and SAT/CV-Bench.
   - That means the repo is more of a specialization layer on top of an established multimodal RL stack than a ground-up framework.

12. **How to draw the architecture**
   - A good diagram would show four boxes: environment/setup, SAT preprocessing, multimodal training package, and evaluation.
   - Inside training, separate the algorithm layer (SFT/GRPO/generation/eval) from the trainer/model adaptation layer.
   - Connect training outputs to both local evaluation and external checkpoint publishing.

Main entry points: [`src/open-r1-multimodal`](https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal) (Open-R1 MM), [`src/open-r1-multimodal/src/open_r1/trainer`](https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal/src/open_r1/trainer) (Trainer), [`setup.sh`](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/setup.sh) (Setup), [`src/eval`](https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/eval) (CVBench eval)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Orchestration"]
  node_readme["README<br/>guide<br/>[README.md]"]
  node_setup["Setup<br/>env bootstrap<br/>[setup.sh]"]
end

subgraph group_data["SAT data"]
  node_sat_root["SAT prep<br/>dataset pipeline"]
  node_sat_script["Prepare data<br/>shell entrypoint<br/>[prepare_dataset.sh]"]
  node_sat_process["Process data<br/>python transform<br/>[process_dataset.py]"]
end

subgraph group_training["Training stack"]
  node_r1_root["Open-R1 MM<br/>training package"]
  node_base_bootstrap["Base model<br/>bootstrap script<br/>[prepare_2B_base.sh]"]
  node_run_grpo["GRPO run<br/>training script<br/>[run_grpo.sh]"]
  node_run_grpo_sat["GRPO SAT<br/>training script<br/>[run_grpo_SAT.sh]"]
  node_run_sft["SFT run<br/>training script<br/>[run_sft.sh]"]
  node_run_sft_sat["SFT SAT<br/>training script<br/>[run_sft_SAT.sh]"]
  node_configs["DDP/ZeRO<br/>distributed config"]
  node_openr1_pkg["open_r1<br/>core training pkg"]
  node_sft_py["SFT core<br/>algorithm module<br/>[sft.py]"]
  node_grpo_py["GRPO core<br/>algorithm module<br/>[grpo.py]"]
  node_gen_py["Generate<br/>rollout module<br/>[generate.py]"]
  node_eval_py["Eval core<br/>evaluation module<br/>[evaluate.py]"]
  node_trainer_pkg["Trainer"]
  node_grpo_trainer["GRPO trainer<br/>custom trainer<br/>[grpo_trainer.py]"]
  node_internvl2["InternVL2<br/>model adapter<br/>[InternVL2.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_cvbench_root["CVBench eval<br/>evaluation pipeline"]
  node_cvbench_base["Base eval<br/>eval script"]
  node_cvbench_instruct["Reason eval<br/>eval script"]
end

node_readme -->|"documents"| node_setup
node_setup -->|"prepares"| node_sat_root
node_setup -->|"installs"| node_r1_root
node_sat_root -->|"contains"| node_sat_script
node_sat_script -->|"runs"| node_sat_process
node_sat_process -->|"feeds"| node_r1_root
node_r1_root -->|"includes"| node_base_bootstrap
node_r1_root -->|"exposes"| node_run_grpo
node_r1_root -->|"exposes"| node_run_grpo_sat
node_r1_root -->|"exposes"| node_run_sft
node_r1_root -->|"exposes"| node_run_sft_sat
node_r1_root -->|"uses"| node_configs
node_run_sft -->|"calls"| node_sft_py
node_run_grpo -->|"calls"| node_grpo_py
node_run_grpo_sat -->|"calls"| node_grpo_py
node_run_sft_sat -->|"calls"| node_sft_py
node_grpo_py -->|"depends on"| node_trainer_pkg
node_sft_py -->|"depends on"| node_trainer_pkg
node_gen_py -->|"uses"| node_trainer_pkg
node_eval_py -->|"uses"| node_trainer_pkg
node_trainer_pkg -->|"implements"| node_grpo_trainer
node_trainer_pkg -->|"adapts"| node_internvl2
node_grpo_trainer -->|"targets"| node_internvl2
node_cvbench_root -->|"contains"| node_cvbench_base
node_cvbench_root -->|"contains"| node_cvbench_instruct
node_r1_root -->|"exports"| node_cvbench_root

click node_readme "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/README.md"
click node_setup "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/setup.sh"
click node_sat_root "https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/data/SAT"
click node_sat_script "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/data/SAT/prepare_dataset.sh"
click node_sat_process "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/data/SAT/process_dataset.py"
click node_r1_root "https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal"
click node_base_bootstrap "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/prepare_2B_base.sh"
click node_run_grpo "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/run_grpo.sh"
click node_run_grpo_sat "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/run_grpo_SAT.sh"
click node_run_sft "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/run_sft.sh"
click node_run_sft_sat "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/run_sft_SAT.sh"
click node_configs "https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal/configs"
click node_openr1_pkg "https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal/src/open_r1"
click node_sft_py "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/sft.py"
click node_grpo_py "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/grpo.py"
click node_gen_py "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/generate.py"
click node_eval_py "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/evaluate.py"
click node_trainer_pkg "https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal/src/open_r1/trainer"
click node_grpo_trainer "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/trainer/grpo_trainer.py"
click node_internvl2 "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/trainer/InternVL2.py"
click node_cvbench_root "https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/eval"
click node_cvbench_base "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/eval/evaluate_Qwen2_VL_CVBench-base.py"
click node_cvbench_instruct "https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/eval/evaluate_Qwen2_VL_CVBench.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_readme,node_setup toneBlue
class node_sat_root,node_sat_script,node_sat_process toneAmber
class node_r1_root,node_base_bootstrap,node_run_grpo,node_run_grpo_sat,node_run_sft,node_run_sft_sat,node_configs,node_openr1_pkg,node_sft_py,node_grpo_py,node_gen_py,node_eval_py,node_trainer_pkg,node_grpo_trainer,node_internvl2 toneMint
class node_cvbench_root,node_cvbench_base,node_cvbench_instruct toneRose
```

## Components

### Orchestration

Repo entrypoints and workflow guidance.

- [README](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/README.md): guide · Workflow overview and usage notes. · `README.md`
- [Setup](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/setup.sh): env bootstrap · Top-level dependency and environment setup entrypoint. · `setup.sh`

### SAT data

SAT dataset ingestion and preprocessing.

- [SAT prep](https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/data/SAT): dataset pipeline · Boundary for SAT dataset preparation. · `src/data/SAT`
- [Prepare data](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/data/SAT/prepare_dataset.sh): shell entrypoint · Runs SAT dataset preprocessing. · `src/data/SAT/prepare_dataset.sh`
- [Process data](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/data/SAT/process_dataset.py): python transform · Transforms raw SAT data into training-ready form. · `src/data/SAT/process_dataset.py`

### Training stack

Reusable multimodal training package and run modes.

- [Open-R1 MM](https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal): training package · Reusable multimodal training codebase. · `src/open-r1-multimodal`
- [Base model](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/prepare_2B_base.sh): bootstrap script · Prepares the 2B base model or training state. · `src/open-r1-multimodal/prepare_2B_base.sh`
- [GRPO run](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/run_grpo.sh): training script · Default reinforcement-learning training entrypoint. · `src/open-r1-multimodal/run_grpo.sh`
- [GRPO SAT](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/run_grpo_SAT.sh): training script · GRPO variant tuned for the SAT task. · `src/open-r1-multimodal/run_grpo_SAT.sh`
- [SFT run](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/run_sft.sh): training script · Supervised fine-tuning entrypoint. · `src/open-r1-multimodal/run_sft.sh`
- [SFT SAT](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/run_sft_SAT.sh): training script · SFT variant tuned for the SAT task. · `src/open-r1-multimodal/run_sft_SAT.sh`
- [DDP/ZeRO](https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal/configs): distributed config · Distributed training and memory partitioning configs. · `src/open-r1-multimodal/configs`
- [open_r1](https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal/src/open_r1): core training pkg · Main algorithm and model-facing Python package. · `src/open-r1-multimodal/src/open_r1`
- [SFT core](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/sft.py): algorithm module · Supervised fine-tuning logic. · `src/open-r1-multimodal/src/open_r1/sft.py`
- [GRPO core](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/grpo.py): algorithm module · Reinforcement-learning training logic. · `src/open-r1-multimodal/src/open_r1/grpo.py`
- [Generate](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/generate.py): rollout module · Inference and rollout generation. · `src/open-r1-multimodal/src/open_r1/generate.py`
- [Eval core](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/evaluate.py): evaluation module · Internal evaluation utilities. · `src/open-r1-multimodal/src/open_r1/evaluate.py`
- [Trainer](https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/open-r1-multimodal/src/open_r1/trainer): trainer layer · Custom trainer and model adaptation boundary. · `src/open-r1-multimodal/src/open_r1/trainer`
- [GRPO trainer](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/trainer/grpo_trainer.py): custom trainer · Main RL trainer implementation. · `src/open-r1-multimodal/src/open_r1/trainer/grpo_trainer.py`
- [InternVL2](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/open-r1-multimodal/src/open_r1/trainer/InternVL2.py): model adapter · Model-specific multimodal adaptation. · `src/open-r1-multimodal/src/open_r1/trainer/InternVL2.py`

### Evaluation

CVBench evaluation entrypoints.

- [CVBench eval](https://github.com/turningpoint-ai/visualthinker-r1-zero/tree/main/src/eval): evaluation pipeline · External evaluation stage for CVBench. · `src/eval`
- [Base eval](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/eval/evaluate_Qwen2_VL_CVBench-base.py): eval script · CVBench evaluation for base-model setup. · `src/eval/evaluate_Qwen2_VL_CVBench-base.py`
- [Reason eval](https://github.com/turningpoint-ai/visualthinker-r1-zero/blob/main/src/eval/evaluate_Qwen2_VL_CVBench.py): eval script · CVBench evaluation for the non-base setup. · `src/eval/evaluate_Qwen2_VL_CVBench.py`

## Connections

- README → Setup: documents · README explains how to bootstrap the repo.
- Setup → SAT prep: prepares · Environment setup enables dataset preparation.
- Setup → Open-R1 MM: installs · Environment setup enables the training package.
- SAT prep → Prepare data: contains · SAT prep boundary contains the shell entrypoint.
- Prepare data → Process data: runs · Shell wrapper invokes Python preprocessing.
- Process data → Open-R1 MM: feeds · Processed SAT data becomes training input.
- Open-R1 MM → Base model: includes · Training package includes base-model bootstrap.
- Open-R1 MM → GRPO run: exposes · Package exposes GRPO workflow entrypoint.
- Open-R1 MM → GRPO SAT: exposes · Package exposes SAT-specific GRPO entrypoint.
- Open-R1 MM → SFT run: exposes · Package exposes SFT workflow entrypoint.
- Open-R1 MM → SFT SAT: exposes · Package exposes SAT-specific SFT entrypoint.
- Open-R1 MM → DDP/ZeRO: uses · Distributed execution is parameterized by configs.
- SFT run → SFT core: calls · SFT script launches core SFT logic.
- GRPO run → GRPO core: calls · GRPO script launches core RL logic.
- GRPO SAT → GRPO core: calls · SAT GRPO script reuses the same GRPO core.
- SFT SAT → SFT core: calls · SAT SFT script reuses the same SFT core.
- GRPO core → Trainer: depends on · GRPO algorithm delegates to trainer layer.
- SFT core → Trainer: depends on · SFT algorithm delegates to trainer layer.
- Generate → Trainer: uses · Rollout generation uses the trainer/model boundary.
- Eval core → Trainer: uses · Internal evaluation utilities depend on the trainer/model boundary.
- Trainer → GRPO trainer: implements · Trainer package contains the RL trainer implementation.
- Trainer → InternVL2: adapts · Trainer package contains model-specific adaptation.
- GRPO trainer → InternVL2: targets · RL training is built around the multimodal backbone adapter.
- CVBench eval → Base eval: contains · Evaluation boundary contains the base-model script.
- CVBench eval → Reason eval: contains · Evaluation boundary contains the non-base script.
- Open-R1 MM → CVBench eval: exports · Trained checkpoints are evaluated by the external CVBench stage.
