How tulerfeng/video-r1 works
Main Components Identification
• Data & Dataset Layer:
• Input datasets include two primary JSON files (Video-R1-260k for RL training and Video-R1-COT-165k for the SFT cold start).
• Data preparation utilities like unzip.py (for uncompressing the dataset) and scripts or instructions for placing data into the proper folder (src/r1-v/Video-R1-data).
• Utility & Preprocessing Layer:
• “qwen-vl-utils”: Contains utility scripts (e.g., vision_process.py) that likely handle video extraction, frame processing (using decord), and other pre-processing activities.
• This module is crucial for converting raw video and image data into the format needed for the training pipelines.
• Model Training Module (Core Compute Engine):
• “r1-v”: Contains the main training pipelines including configurations, trainers, and scripts for both SFT and RL training.
• Inside r1-v, different configurations (ddp.yaml, qwen2vl_sft_config.yaml, etc.) and local scripts (for both SFT and RL) indicate that this component handles the training setup, hyperparameter configurations, and execution commands.
• Distinguish between SFT (using the run_sft_video.sh script) and RL training (using run_grpo_video.sh and run_grpo_vllm_qwen25vl.sh). Clearly indicate that SFT is the initial phase (coarse tuning) and RL (T-GRPO extension) is applied afterwards.
• Evaluation & Inference Module:
• Located in the “eval” folder, with additional evaluation scripts and JSONL prompts.
• Includes scripts (like eval_bench.sh) for running inference on various evaluation benchmarks and for processing evaluation video data.
• The eval component communicates with the trained model and utilizes external benchmark data.
• Auxiliary Components:
• “scripts” present in both the src folder and previous_version subdirectories that tie together various training and evaluation commands.
• Configuration and dependency management files (setup.sh, requirements files, pyproject.toml) that aid in environment setup and reproducibility.
• “previous_version” folder which shows an earlier iteration of the project – a useful reference for understanding the development evolution.
Relationships & Interactions
• Data flows from the dataset depot (raw JSON and video/image data) through preprocessing (qwen-vl-utils) where videos are extracted and prepared.
• Preprocessed data is then fed into the training pipeline under r1-v:
• The SFT component uses the prepared “Video-R1-COT-165k” data to fine-tune an initial model (via run_sft_video.sh).
• Following SFT, RL training is carried out on “Video-R1-260k” with temporal reasoning extensions (T-GRPO), where the model’s performance is validated with specialized scripts.
• The trained model is then evaluated using the eval scripts and benchmark JSON configurations.
• External dependencies such as the specific versions of Transformers, vLLM, and trl libraries are also key; include them as dependencies feeding into both training and inference.
Architectural Patterns & Design Principles
• Layered Architecture: Separate modules for data ingestion, preprocessing, training, and evaluation.
• Pipeline Orchestration: The sequential flow from dataset processing to supervised training and then reinforcement learning demonstrates a clear pipeline design.
• Modularity & Extensibility: With distinct directories for “qwen-vl-utils” and “r1-v”, the project is modular; each can be updated or replaced without completely disrupting the system.