How turningpoint-ai/visualthinker-r1-zero works
1. Repository purpose
- This repo is an end-to-end pipeline for reproducing a multimodal RL result: training a 2B vision-language model to show an “aha moment” on SAT-style visual reasoning, then evaluating it on CVBench.
- The architecture is organized around three stages: environment/setup, dataset preparation, and model training/evaluation.
2. Top-level orchestration
- setup.sh is the entrypoint for installing dependencies and preparing the environment.
- The root README.md acts as the workflow guide, but the actual implementation is split into src/data, src/open-r1-multimodal, and src/eval.
3. Data preparation subsystem
- src/data/SAT/prepare_dataset.sh and process_dataset.py form the dataset pipeline for the SAT task.
- This is the ingestion boundary: raw SAT data is transformed into a training-ready format before any model code runs.
- The repo does not appear to contain the dataset itself, only the preparation logic.
4. Training framework boundary
- src/open-r1-multimodal is a self-contained training package, separate from the dataset and evaluation code.
- It includes its own setup.py, setup.cfg, Makefile, and Python package under src/open_r1, indicating this is the main reusable training codebase rather than a one-off script collection.
5. Model training workflows
- Training is exposed through shell entrypoints: run_grpo.sh, run_grpo_SAT.sh, run_sft.sh, and run_sft_SAT.sh.
- These scripts select the training mode and task variant:
- GRPO for reinforcement-learning-based reasoning training
- SFT for supervised fine-tuning comparisons
- SAT versions for the visual reasoning dataset used in the paper
- prepare_2B_base.sh suggests a separate bootstrap step for initializing the 2B base model or its training state.
6. Core algorithm implementations
- In src/open_r1, the major training/evaluation logic is split by capability:
- sft.py for supervised fine-tuning
- grpo.py for GRPO training
- generate.py for inference / rollout generation
- evaluate.py for evaluation utilities
- This indicates a modular design where data sampling, rollout generation, optimization, and evaluation are separated.
7. Trainer layer
- src/open_r1/trainer/grpo_trainer.py is the main custom trainer implementation for RL-style optimization.
- src/open_r1/trainer/InternVL2.py suggests model-specific adaptation for a multimodal backbone or chat format.
- The trainer layer is the boundary between the general training algorithms and the specific multimodal model implementation.
8. Distributed training infrastructure
- configs/ddp.yaml, configs/zero2.yaml, and configs/zero3.yaml indicate support for distributed execution and DeepSpeed-style ZeRO optimization levels.
- This matters architecturally because training is designed to run on multi-GPU systems and the config choice changes memory partitioning and parallelization behavior.
9. Evaluation subsystem
- src/eval/evaluate_Qwen2_VL_CVBench.py and evaluate_Qwen2_VL_CVBench-base.py are separate evaluation entrypoints for different model variants.
- The split between base and non-base scripts reflects two model families or prompting setups:
- base model vs. instruct-tuned model
- with or without reasoning prompt
- Evaluation is an externalized stage, not embedded in training.
10. End-to-end data flow
- Flow is: raw SAT data → dataset processing → GRPO/SFT training on the open-r1 multimodal package → saved checkpoints → CVBench evaluation scripts.
- The trained checkpoint is also published externally to Hugging Face, so model artifacts are an output boundary of the repo.
11. External dependencies and references that shape the architecture
- The project is built around existing multimodal and RL foundations: Qwen2-VL, Open-R1, Open-R1-Multimodal, DeepSeek-R1, and SAT/CV-Bench.
- That means the repo is more of a specialization layer on top of an established multimodal RL stack than a ground-up framework.
12. How to draw the architecture
- A good diagram would show four boxes: environment/setup, SAT preprocessing, multimodal training package, and evaluation.
- Inside training, separate the algorithm layer (SFT/GRPO/generation/eval) from the trainer/model adaptation layer.
- Connect training outputs to both local evaluation and external checkpoint publishing.
Main entry points: open-r1-multimodal Open-R1 MM · trainer Trainer · setup.sh Setup · eval CVBench eval
Components 23
Orchestration
Repo entrypoints and workflow guidance.
SAT data
SAT dataset ingestion and preprocessing.
- SAT prep · Boundary for SAT dataset preparation.
src/data/SAT - Prepare data · Runs SAT dataset preprocessing.
src/data/SAT/prepare_dataset.sh - Process data · Transforms raw SAT data into training-ready form.
src/data/SAT/process_dataset.py
Training stack
Reusable multimodal training package and run modes.
- Open-R1 MM · Reusable multimodal training codebase.
src/open-r1-multimodal - Base model · Prepares the 2B base model or training state.
src/open-r1-multimodal/prepare_2B_base.sh - GRPO run · Default reinforcement-learning training entrypoint.
src/open-r1-multimodal/run_grpo.sh - GRPO SAT · GRPO variant tuned for the SAT task.
src/open-r1-multimodal/run_grpo_SAT.sh - SFT run · Supervised fine-tuning entrypoint.
src/open-r1-multimodal/run_sft.sh - SFT SAT · SFT variant tuned for the SAT task.
src/open-r1-multimodal/run_sft_SAT.sh - DDP/ZeRO · Distributed training and memory partitioning configs.
src/open-r1-multimodal/configs - open_r1 · Main algorithm and model-facing Python package.
src/open-r1-multimodal/src/open_r1 - SFT core · Supervised fine-tuning logic.
src/open-r1-multimodal/src/open_r1/sft.py - GRPO core · Reinforcement-learning training logic.
src/open-r1-multimodal/src/open_r1/grpo.py - Generate · Inference and rollout generation.
src/open-r1-multimodal/src/open_r1/generate.py - Eval core · Internal evaluation utilities.
src/open-r1-multimodal/src/open_r1/evaluate.py - Trainer · Custom trainer and model adaptation boundary.
src/open-r1-multimodal/src/open_r1/trainer - GRPO trainer · Main RL trainer implementation.
src/open-r1-multimodal/src/open_r1/trainer/grpo_trainer.py - InternVL2 · Model-specific multimodal adaptation.
src/open-r1-multimodal/src/open_r1/trainer/InternVL2.py
Evaluation
CVBench evaluation entrypoints.
- CVBench eval · External evaluation stage for CVBench.
src/eval - Base eval · CVBench evaluation for base-model setup.
src/eval/evaluate_Qwen2_VL_CVBench-base.py - Reason eval · CVBench evaluation for the non-base setup.
src/eval/evaluate_Qwen2_VL_CVBench.py
Connections 26
- README → Setup: documents · README explains how to bootstrap the repo.
- Setup → SAT prep: prepares · Environment setup enables dataset preparation.
- Setup → Open-R1 MM: installs · Environment setup enables the training package.
- SAT prep → Prepare data: contains · SAT prep boundary contains the shell entrypoint.
- Prepare data → Process data: runs · Shell wrapper invokes Python preprocessing.
- Process data → Open-R1 MM: feeds · Processed SAT data becomes training input.
- Open-R1 MM → Base model: includes · Training package includes base-model bootstrap.
- Open-R1 MM → GRPO run: exposes · Package exposes GRPO workflow entrypoint.
- Open-R1 MM → GRPO SAT: exposes · Package exposes SAT-specific GRPO entrypoint.
- Open-R1 MM → SFT run: exposes · Package exposes SFT workflow entrypoint.
- Open-R1 MM → SFT SAT: exposes · Package exposes SAT-specific SFT entrypoint.
- Open-R1 MM → DDP/ZeRO: uses · Distributed execution is parameterized by configs.
- SFT run → SFT core: calls · SFT script launches core SFT logic.
- GRPO run → GRPO core: calls · GRPO script launches core RL logic.
- GRPO SAT → GRPO core: calls · SAT GRPO script reuses the same GRPO core.
- SFT SAT → SFT core: calls · SAT SFT script reuses the same SFT core.
- GRPO core → Trainer: depends on · GRPO algorithm delegates to trainer layer.
- SFT core → Trainer: depends on · SFT algorithm delegates to trainer layer.
- Generate → Trainer: uses · Rollout generation uses the trainer/model boundary.
- Eval core → Trainer: uses · Internal evaluation utilities depend on the trainer/model boundary.
- Trainer → GRPO trainer: implements · Trainer package contains the RL trainer implementation.
- Trainer → InternVL2: adapts · Trainer package contains model-specific adaptation.
- GRPO trainer → InternVL2: targets · RL training is built around the multimodal backbone adapter.
- CVBench eval → Base eval: contains · Evaluation boundary contains the base-model script.
- CVBench eval → Reason eval: contains · Evaluation boundary contains the non-base script.
- Open-R1 MM → CVBench eval: exports · Trained checkpoints are evaluated by the external CVBench stage.