# deep-agent/r1-v architecture

> Architecture diagram of the GitHub repository deep-agent/r1-v, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deep-agent/r1-v
- Repository: https://github.com/deep-agent/r1-v

## Overview

##### Project Type and Purpose
- The project is a research-oriented open‐source framework for enhancing Vision Language Models (VLMs) through reinforcement learning and supervised fine-tuning.
- Its purpose is to experiment with and improve general visual reasoning (e.g., counting and geometry reasoning) on established datasets (CLEVR, GEOQA) using methods like GRPO and SFT.

##### Analyzing the File Structure
- Top-level files/directories:
– .gitignore, README.md, setup.sh – for basic project configuration.
– images – for assets used in the documentation.
– src directory – which contains the main source code.
- Within src, there are several key submodules:
a. src/distill_r1:
- Contains scripts and notebooks (e.g., generate_scene_qa_pairs.ipynb, prompt.py, query_r1.py) designed for data distillation and dataset creation.
b. src/eval:
- Houses evaluation scripts, logs, and prompt files. This module is clearly dedicated to assessing the trained models on tasks such as SuperCLEVR and GEOQA.
c. src/r1-v:
- This is the core training and experimentation engine. It includes:
▪ Configuration files (in the configs directory) for different training setups (ddp, zero configs, qwen2vl_sft_config).
▪ Local scripts to generate data, prepare training, and run experiments.
▪ Main modules in src/open_r1 with files for evaluation (evaluate.py), generation (generate.py), reinforcement learning policy optimization (grpo.py), and supervised fine-tuning (sft.py).
▪ Trainer implementations including GRPO and modified VLLM-based trainers in the trainer subdirectory.
d. src/scripts:
- Contains convenient shell scripts to execute various experiments (e.g., running GRPO with different configurations, SFT, tests for specific datasets).

##### Extracting Insights from the README
- The README emphasizes two major training paradigms: GRPO (reinforcement learning) and SFT (supervised fine tuning).
- It lists supported models (Qwen2-VL, Qwen2.5-VL) and details datasets including CLEVR-70k and GEOQA-8k.
- It provides command-line examples with torchrun (distributed training) and accelerate (for SFT), which suggests distributed setups and GPU/multi-node usage.
- External resources (e.g., HuggingFace datasets, Git LFS for large images) also play a role.

a. Main Components
- Data Preparation & Distillation Module:
▪ Represented by src/distill_r1.
▪ Responsible for dataset generation, filtering, and distillation.
- Core Training Engine:
▪ Represented by src/r1-v.
▪ Contains submodules for RL-based training (GRPO) and SFT.
▪ Includes configurations, local scripts, and trainer implementations.
- Evaluation & Testing Module:
▪ Represented by src/eval and related scripts.
▪ Handles testing (SuperCLEVR and GEOQA), log generation, and result analysis.
- Utility and Support Scripts:
▪ Represented by setup.sh, src/scripts, and configuration files.
▪ They handle environment setup, command orchestration, and deployment scripts.
- External Integration:
▪ Datasets from HuggingFace, external images (using Git LFS), and possibly logging/visualization platforms like WandB.
- Distributed/Parallel Training Infrastructure:
▪ Incorporate torchrun, accelerate, DeepSpeed, and vLLM support that tie into the core training engine.

b. Relationships & Interactions
- Data flows from the Data Preparation Module to the Core Training Engine.
- The Core Training Engine outputs models trained via either GRPO or SFT.
- The Evaluation Module takes the trained models as input, runs tests (counting, geometry reasoning), and outputs logs/results.
- Utility scripts and configuration files interface with both training and evaluation pipelines.
- External datasets and resources are fed into the Data Preparation Module and referenced by training/evaluation scripts.

c. Architectural Patterns & Design Principles
- Separation of Concerns:
▪ Data distillation, training, and evaluation are managed in their own subdirectories.
- Modular Design:
▪ Common configurations and shared modules (such as open_r1) that are reused across different experiments.
- Distributed & Scalable Training:
▪ Use of distributed training commands (torchrun, DeepSpeed, vLLM) to support multi-GPU setups.
- Experimentation & Extensibility:
▪ Use of separate local scripts and configuration files that make it straightforward to plug in new datasets or training methods.

d. Relevant Technologies, Frameworks, or Libraries
- Python 3.11 (as per setup.sh and conda instructions)
- PyTorch (distributed training via torchrun)
- DeepSpeed (as shown by the JSON config files)
- vLLM and accelerate for optimized training and GPU utilization
- Jupyter Notebook (as indicated by *.ipynb files for dataset generation)
- Git LFS for managing large data (images)
- WandB for logging experiments

##### Elements
- From External Data Sources → Data Distillation.
- Data Distillation → Training Engine.
- Training Engine → Evaluation.
- Differentiate between computation nodes (training modules), data repositories (datasets, logs), and orchestration scripts.

## Diagram (Mermaid)

```mermaid
flowchart TD
    E["External Data/Services"]:::external

    subgraph "Data Preparation & Distillation"
        DP1["Data Distillation Module"]:::module
    end

    subgraph "Core Training (GRPO & SFT)"
        CT1["Training Engine"]:::module
        CT2["Distributed Training (torchrun/accelerate/vLLM)"]:::note
    end

    subgraph "Evaluation & Testing"
        EV1["Evaluation Module"]:::module
    end

    subgraph "Utility & Support"
        U1["Setup Script"]:::utility
        U2["Utility Scripts"]:::utility
    end

    E -->|"dataInput"| DP1
    DP1 -->|"datasetOutput"| CT1
    CT1 -->|"modelOutput"| EV1
    U1 -->|"configControl"| CT1
    U2 -->|"experimentOrchestration"| CT1
    U2 -->|"experimentOrchestration"| EV1
    EV1 -->|"feedback"| CT1

    click DP1 "https://github.com/deep-agent/r1-v/tree/main/src/distill_r1"
    click CT1 "https://github.com/deep-agent/r1-v/tree/main/src/r1-v"
    click EV1 "https://github.com/deep-agent/r1-v/tree/main/src/eval"
    click U1 "https://github.com/deep-agent/r1-v/blob/main/setup.sh"
    click U2 "https://github.com/deep-agent/r1-v/tree/main/src/scripts"

    classDef external fill:#FFEEAD,stroke:#333,stroke-width:2px,color:#000;
    classDef module fill:#AED6F1,stroke:#333,stroke-width:2px,color:#000;
    classDef note fill:#A9DFBF,stroke:#333,stroke-width:2px,color:#000;
    classDef utility fill:#F9D5E5,stroke:#333,stroke-width:2px,color:#000;
```
