# open-thought/reasoning-gym architecture

> Architecture diagram of the GitHub repository open-thought/reasoning-gym, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/open-thought/reasoning-gym
- Repository: https://github.com/open-thought/reasoning-gym
- Video tour (about a minute): https://gitdiagram.com/open-thought/reasoning-gym/video

## Overview

Based on our analysis of the file tree and the README, “Reasoning Gym” is a community-driven Python library that provides a wide range of procedural dataset generators and algorithmically verifiable reasoning tasks. It is designed for training reasoning models (particularly with RL) by dynamically creating and verifying datasets across domains such as algebra, arithmetic, logic, cognition, games, and more.

##### Main Components
- Core Library (reasoning_gym): This is the heart of the project and includes all the task generators. It covers various domains (algebra, algorithmic, arc, arithmetic, coaching, code, cognition, games, geometry, graphs, induction, logic, etc.) and is implemented as modular sub-packages.
- Evaluation Module (eval directory): Supports automated evaluation and visualization. It includes scripts (e.g., eval.py, visualize_results.py) and YAML configurations for different models and evaluation profiles.
- Training Infrastructure (training directory): Contains scripts and configuration files that set up the training routines (e.g., train_grpo.py, trainer modules) which would interface with RL frameworks.
- Command-Line Tools and Server (tools directory): Includes CLI tools and a server component, each with their configuration and test scripts, to enable user interactions, possibly for deployment or for accessing the models/datasets via network APIs.
- Supporting Components: Tests, notebooks, scripts, and CI/CD configurations (e.g., GitHub workflows, pre-commit hooks) all support project quality, integration, and continuous deployment.

##### Relationships and Interactions
- The Core Library (reasoning_gym) provides the API for dataset generation. Client code (whether from examples, training routines, or external users via pip-installed package) calls functions like create_dataset.
- The Evaluation module consumes datasets and model outputs – using evaluation configuration files (YAML and JSON examples) to govern evaluations. Play out the data flow: dataset generation → model execution → evaluation/verification → visualization.
- Training routines use trainers (in training/trainers) to couple with RL algorithms and adapt dataset generators from the core library.
- The example projects are “client” applications illustrating how to integrate the library with external systems.
- CI/CD configurations ensure consistent testing and integration runs when modifications are made.

##### Architectural Patterns and Design Principles
- Modularity: The project is split into various domains (for dataset generation, evaluation, training, and examples) that interact via well-defined interfaces. Illustrate this by drawing modules as independent blocks with clear connections.
- Separation of Concerns: Core dataset generation logic is separated from evaluation, training, and user-interface scripts/CLI tools.
- Reusability and Extensibility: The use of configuration files (often in YAML and JSON) and the standard “score_answer” method for algorithmic verification make it straightforward to add new tasks or extend evaluation schemes.
- Test-Driven Design: A comprehensive tests directory indicates an emphasis on verifiability and correctness.

##### Relevant Technologies, Frameworks, or Libraries
- Primary Language: Python (≥ 3.11)
- Build and Dependency Management: Managed via pip and the pyproject.toml file.
- Evaluation/Visualization: Involves Python scripts and notebooks for visualizing results.
- CI/CD: GitHub workflows (.github/workflows) ensure quality and continuous integration.

- For the Training component, show how training scripts interface with both the core library and external RL frameworks.
- Side components like notebooks (for experimentation), tests (for validation), and CI/CD workflows (for quality assurance).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Library
    subgraph "Core Library"
        RLCL["Reasoning Gym Core Library"]:::core
        subgraph "Dataset & Task Generators"
            A["Algebra"]:::sub
            B["Algorithmic"]:::sub
            C["Arc"]:::sub
            D["Arithmetic"]:::sub
            E["Coaching"]:::sub
            F["Code"]:::sub
            G["Cognition"]:::sub
            H["Games"]:::sub
            I["Geometry"]:::sub
            J["Graphs"]:::sub
            K["Induction"]:::sub
            L["Logic"]:::sub
        end
    end

    %% Evaluation Module
    subgraph "Evaluation Module"
        Eval["Evaluation Engine"]:::eval
    end

    %% Training Infrastructure
    subgraph "Training Infrastructure"
        Training["Training Module"]:::training
    end

    %% Example Applications
    subgraph "Example Applications"
        Ex1["OpenRLHF"]:::examples
        Ex2["open-instruct"]:::examples
        Ex3["trl"]:::examples
        Ex4["veRL"]:::examples
        Ex5["word_ladder"]:::examples
    end

    %% Command-Line Tools and Server
    subgraph "Command-Line Tools and Server"
        ToolsCLI["CLI Tools"]:::tools
        ToolsServer["Server Tools"]:::tools
    end

    %% Supporting Components
    subgraph "Supporting Components"
        Tests["Tests"]:::support
        Notebooks["Notebooks"]:::support
        CICD["CI/CD"]:::support
    end

    %% Relationships
    RLCL -->|"dataset_gen"| Eval
    RLCL -->|"dataset_gen"| Training
    RLCL -->|"used_in"| Ex1
    RLCL -->|"used_in"| Ex2
    RLCL -->|"used_in"| Ex3
    RLCL -->|"used_in"| Ex4
    RLCL -->|"used_in"| Ex5
    RLCL -->|"cli_access"| ToolsCLI
    RLCL -->|"server_access"| ToolsServer
    RLCL -->|"tested_by"| Tests
    Eval -->|"visualizes_via"| Notebooks
    Training -->|"config_control"| CICD

    %% Click Events
    click RLCL "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/"
    click A "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/algebra"
    click B "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/algorithmic"
    click C "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/arc"
    click D "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/arithmetic"
    click E "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/coaching"
    click F "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/code"
    click G "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/cognition"
    click H "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/games"
    click I "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/geometry"
    click J "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/graphs"
    click K "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/induction"
    click L "https://github.com/open-thought/reasoning-gym/tree/main/reasoning_gym/logic"
    click Eval "https://github.com/open-thought/reasoning-gym/tree/main/eval/"
    click Training "https://github.com/open-thought/reasoning-gym/tree/main/training/"
    click Ex1 "https://github.com/open-thought/reasoning-gym/tree/main/examples/OpenRLHF"
    click Ex2 "https://github.com/open-thought/reasoning-gym/tree/main/examples/open-instruct"
    click Ex3 "https://github.com/open-thought/reasoning-gym/tree/main/examples/trl"
    click Ex4 "https://github.com/open-thought/reasoning-gym/tree/main/examples/veRL"
    click Ex5 "https://github.com/open-thought/reasoning-gym/tree/main/examples/word_ladder"
    click ToolsCLI "https://github.com/open-thought/reasoning-gym/tree/main/tools/cli/"
    click ToolsServer "https://github.com/open-thought/reasoning-gym/tree/main/tools/server/"
    click Tests "https://github.com/open-thought/reasoning-gym/tree/main/tests/"
    click Notebooks "https://github.com/open-thought/reasoning-gym/tree/main/notebooks/"
    click CICD "https://github.com/open-thought/reasoning-gym/tree/main/.github/workflows/"

    %% Styles
    classDef core fill:#ffcccc,stroke:#b30000,stroke-width:2px;
    classDef sub fill:#ffe6e6,stroke:#b30000,stroke-width:2px;
    classDef eval fill:#ccffcc,stroke:#006600,stroke-width:2px;
    classDef training fill:#cce6ff,stroke:#004d99,stroke-width:2px;
    classDef examples fill:#ffffcc,stroke:#b3b300,stroke-width:2px;
    classDef tools fill:#e6ccff,stroke:#663399,stroke-width:2px;
    classDef support fill:#d9d9d9,stroke:#808080,stroke-width:2px;
```
