How zzzhhzzz/ground-r1 works
Project type and purpose
• This is a machine-learning research codebase implementing “Ground-R1,” a reinforcement-learning framework for grounded visual reasoning with large vision-language models (LVLMs).
• It provides:
– A reinforcement learning (RL) training pipeline that generates evidence regions and answers.
– A supervised fine-tuning (SFT) pipeline for comparison.
– Evaluation scripts across multiple vision-language benchmarks (VisCoT, MME, RefCOCO, etc.).
File structure
Top level directories and their roles:
• asset: static images for README visualization
• eval: evaluation scripts & datasets specifically for VisCoT and RefCOCO
• eval_general_dataset: evaluation across general LVLM benchmarks (MME, SEED-Bench…)
• qwen-vl-finetune: supervised fine-tuning module (data prep, training code)
• r1-v: core RL training code, configurations, model implementation
• train_data: scripts and raw data for initial dataset preparation
• scripts, setup.sh: orchestration scripts for end-to-end runs
Key subcomponents:
– r1-v/src/open_r1: Python implementation of RL environment, reward functions, trainers
– r1-v/configs: YAML files driving hyperparameters for different runs (zero2, zero3, qwen2vl_sft)
– qwen-vl-finetune/qwenvl/train: SFT trainer, argument parsing, training loops
– eval & eval_general_dataset: multiple .sh drivers invoking Python scripts for metrics calculation
README for additional insights
• Describes two phases: grounding (evidence region rollout) and answering (reward-guided responses).
• Lists dependencies (Conda, Python 3.10, external datasets on Hugging Face).
• Details training & inference commands, plus evaluation across benchmarks.
a. Main components
1. Data Sources
– Visual-CoT dataset
– LVLM benchmark datasets (MME, SEED-Bench…)
– RefCOCO family datasets
2. Preprocessing
– Data preparation scripts (train_data/prepare_data.py, qwen-vl-finetune/scripts/prepare_SFT_data.sh)
3. Model & Training
– RL Trainer module (r1-v/src/open_r1/trainer)
– Reward functions & environment (format reward, accuracy reward)
– SFT Trainer module (qwen-vl-finetune/qwenvl/train)
4. Configuration Management
– YAML config files (r1-v/configs/*.yaml)
5. Evaluation
– VisCoT & RefCOCO eval scripts (eval/, eval_general_dataset/)
6. Orchestration & Deployment
– Shell scripts (scripts/run_grpo_video.sh, multi_*.sh)
– Environment setup (setup.sh, conda environment)
b. Relationships and data flow
1. Raw datasets → Preprocessing → Formatted data
2. Formatted data + Config → Training module (RL or SFT)
3. Trained model checkpoints → Evaluation scripts → Metrics & plots
4. Optional: Checkpoints & logs stored for analysis
c. Architectural patterns & principles
• Modular ML pipeline: data, training, evaluation isolated in folders
• Config-driven runs: swap YAML files for different experiments
• Two-phase decoupling: evidence grounding vs. answer generation
• Shell orchestration for reproducibility
d. Key technologies & frameworks
• Python 3.10, PyTorch (implied by trainer.py)
• vLLM for efficient language model serving (in trainer modules)
• Hugging Face Datasets for data loading
• YAML for configuration (r1-v/configs/)
• Bash scripts for orchestration
Elements
• Clear labels: e.g., “Data Prep,” “RL Trainer,” “Reward Functions,” “Eval VisCoT.”
– Data sources (green)
– Processing & training modules (blue)
– Configuration (yellow)
– Evaluation (orange)
Final recommendation
– Be exhaustive: represent each directory as a component or subcomponent.
– Capture the two-phase RL pattern visually.