# knoveleng/open-rs architecture

> Architecture diagram of the GitHub repository knoveleng/open-rs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/knoveleng/open-rs
- Repository: https://github.com/knoveleng/open-rs

## Overview

##### Project Type and Purpose
- This is a research-oriented Python library and CLI tool (“Open RS”) for fine-tuning small LLMs (1.5B params) via reinforcement learning (GRPO) and evaluating them on math reasoning tasks.
- No UI/frontend: it’s a command-line driven full “ML pipeline” consisting of configuration, training, evaluation, logging, and model/dataset orchestration with Hugging Face and Weights & Biases.

##### File Structure Analysis
- Top-level scripts & configs:
- setup.py / setup.cfg – package install
- eval.sh – shell wrapper for evaluation
- recipes/accelerate_configs/*.yaml – distributed‐training setups
- recipes/grpo.yaml, data_cleaner.yaml – experiment configs
- Source code: `src/open_r1/`
- grpo.py – RL training entry point
- evaluate.py – evaluation logic (via lighteval)
- generate.py – generation utilities
- sft.py – supervised fine-tuning baseline
- utils/ – callbacks, logging (wandb), model/dataset loading, Hugging Face hub I/O
- Output & logs:
- logs/ – hierarchical experiment results (JSON)
- assets/ – figures for README
- Dependencies & infra: uses `accelerate`, `vLLM`, `flash-attn`, `torch`, `wandb`, `huggingface_hub`.

##### Key Architectural Components
- A. Configuration Layer
- YAML configs (recipes/) specify experiment parameters (algorithm, resources, dataset paths).
- accelerate launch CLI orchestrates multi-GPU processes.
- B. Training Engine (GRPO)
- grpo.py imports configs, initializes model+tokenizer, data loader, RL loop.
- Under the hood: vLLM model server, FlashAttention kernels.
- Uses callbacks for checkpointing, wandb logging.
- C. Evaluation Engine
- evaluate.py interfaces with lighteval to run LLM on custom tasks.
- eval.sh orchestrates single/multi-GPU runs, writes JSON results under logs/evals.
- D. Data & Models
- Data: Hugging Face Datasets (open-s1, open-rs, open-deepscaler).
- Models: pull/push via Hugging Face Hub (Open-RS1/2/3, OpenRS-GRPO).
- E. Logging & Visualization
- Wandb for real-time metrics; local JSON logs for batch analyses.
- assets/ holds static plots.

##### Relationships & Data Flow
1. User runs `accelerate launch --config_file recipes/accelerate_configs/... src/open_r1/grpo.py --config recipes/grpo.yaml`
→ Config parser → Initializes training engine
→ Data loader fetches dataset (Hugging Face hub)
→ Model instantiation (vLLM/PyTorch)
→ RL loop emits metrics → wandb + local JSON logs → model checkpoints (Hugging Face hub)
2. For evaluation: user runs `eval.sh` or a direct `lighteval vllm ... src/open_r1/evaluate.py`
→ load trained model → custom lighteval tasks → generation results → JSON under logs/evals
3. Post-processing / overall metrics: logs/overall folders aggregate experiment runs for visualization in assets/.

##### Architectural Patterns & Principles
- Modular CLI pattern: separate entry points for training, evaluation, generation, SFT.
- Config-Driven Execution: YAML + accelerate for reproducibility and distributed scalability.
- Callback-Based Logging: plug wandb, checkpointing without cluttering core algorithm.
- Clear separation: Data ingestion → Model compute → Logging/Monitoring → Hub I/O.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Configuration Layer
    subgraph "Configuration Layer"
        direction TB
        ConfigRecipes["YAML Configs (recipes/)"]:::config
        ConfigParser["Config Manager (src/open_r1/configs.py)"]:::config
    end

    %% Training Workflow
    subgraph "Training Workflow"
        direction TB
        TrainingEngine["Training Engine (grpo.py)"]:::compute
        RLLoop["RL Loop (vLLM + FlashAttention)"]:::compute
        Rewards["Rewards Module (rewards.py)"]:::compute
    end

    %% Evaluation Workflow
    subgraph "Evaluation Workflow"
        direction TB
        EvalEngine["Evaluation Engine (evaluate.py)"]:::compute
        EvalUtils["Evaluation Utils (utils/evaluation.py)"]:::compute
        EvalShell["Eval Shell Script (eval.sh)"]:::compute
    end

    %% Utility Modules
    subgraph "Utility Modules"
        direction TB
        Callbacks["Callbacks (utils/callbacks.py)"]:::utility
        WandbLogging["Wandb Logging (utils/wandb_logging.py)"]:::utility
        HubUtils["Hub I/O (utils/hub.py)"]:::utility
        ModelUtils["Model Utils (utils/model_utils.py)"]:::utility
        ImportUtils["Import Utils (utils/import_utils.py)"]:::utility
    end

    %% Generation & SFT
    Generate["Generation Utils (generate.py)"]:::compute
    SFT["SFT Baseline (sft.py)"]:::compute

    %% External Services & Storage
    DataStore["HF Datasets"]:::external
    ModelStore["HF Hub"]:::external
    Wandb["Weights & Biases"]:::external
    LogsDir["Local Logs (logs/)"]:::storage
    EvalsDir["Eval Outputs (logs/evals/)"]:::storage
    OverallDir["Aggregated Metrics (logs/overall/)"]:::storage
    Assets["Assets (assets/)"]:::storage

    %% Arrows / Data Flow
    ConfigRecipes -->|"reads"| ConfigParser
    ConfigParser -->|"accelerate launch"| TrainingEngine
    TrainingEngine -->|"RL loop init"| RLLoop
    RLLoop -->|"compute gradients"| Rewards
    TrainingEngine -->|"read samples"| DataStore
    TrainingEngine -->|"push/pull models"| ModelStore
    TrainingEngine -->|"wandb logging"| Wandb
    TrainingEngine -->|"JSON logs"| LogsDir
    TrainingEngine -->|"checkpoint via HF API"| ModelStore

    ConfigParser -->|"reads"| EvalEngine
    EvalShell -->|"launch eval"| EvalEngine
    EvalEngine -->|"load model"| ModelStore
    EvalEngine -->|"run tasks"| EvalUtils
    EvalEngine -->|"write JSON"| EvalsDir

    %% Utilities connections
    TrainingEngine --> Callbacks
    TrainingEngine --> WandbLogging
    TrainingEngine --> HubUtils
    TrainingEngine --> ModelUtils
    TrainingEngine --> ImportUtils
    EvalEngine --> ImportUtils

    Generate --> ImportUtils
    SFT --> ImportUtils

    %% Storage visuals
    LogsDir --> OverallDir
    LogsDir --> EvalsDir
    OverallDir --> Assets

    %% Click Events
    click ConfigRecipes "https://github.com/knoveleng/open-rs/tree/main/recipes/accelerate_configs/"
    click ConfigParser "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/configs.py"
    click TrainingEngine "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/grpo.py"
    click Rewards "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/rewards.py"
    click EvalEngine "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/evaluate.py"
    click EvalShell "https://github.com/knoveleng/open-rs/blob/main/eval.sh"
    click EvalUtils "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/utils/evaluation.py"
    click Callbacks "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/utils/callbacks.py"
    click WandbLogging "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/utils/wandb_logging.py"
    click HubUtils "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/utils/hub.py"
    click ModelUtils "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/utils/model_utils.py"
    click ImportUtils "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/utils/import_utils.py"
    click Generate "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/generate.py"
    click SFT "https://github.com/knoveleng/open-rs/blob/main/src/open_r1/sft.py"
    click LogsDir "https://github.com/knoveleng/open-rs/tree/main/logs/"
    click EvalsDir "https://github.com/knoveleng/open-rs/tree/main/logs/evals/"
    click OverallDir "https://github.com/knoveleng/open-rs/tree/main/logs/overall/"
    click Assets "https://github.com/knoveleng/open-rs/tree/main/assets/"
    click SetupPy "https://github.com/knoveleng/open-rs/blob/main/setup.py"
    click SetupCfg "https://github.com/knoveleng/open-rs/blob/main/setup.cfg"
    click Readme "https://github.com/knoveleng/open-rs/blob/main/README.md"

    %% Styles
    classDef compute fill:#D0E8FF,stroke:#0366D6,stroke-width:1px
    classDef external fill:#E2F0D9,stroke:#237A00,stroke-width:1px,stroke-dasharray: 5 5
    classDef config fill:#FFE8B0,stroke:#D97706,stroke-width:1px
    classDef utility fill:#F3F4F6,stroke:#6B7280,stroke-width:1px
    classDef storage fill:#E5E7EB,stroke:#4B5563,stroke-width:1px,stroke-dasharray: 2 2
```
