# thunlp/schemareinforcementlearning architecture

> Architecture diagram of the GitHub repository thunlp/schemareinforcementlearning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thunlp/schemareinforcementlearning
- Repository: https://github.com/thunlp/schemareinforcementlearning

## Overview

##### Main Components
- Data & Schema Layer: Illustrate the raw schema data (downloaded from JSON Schema Store and GitHub) and the organizational structure (schemabench/data & train/data folders). Indicate that data includes both “custom” and “complex” schemas along with translation tests.
- Data Preprocessing & Parsing: Create a block for the SyntaxParser module. This is responsible for handling different file types (CSV, HTML, JSON, YAML, etc.) and cleaning/parsing the schemas.
- Benchmark & Evaluation Modules: Distinguish two main evaluation components:
– The bench module for running evaluation cases (including benchmarks such as GSM8K, MATH500, MMLU, etc.).
– The berkeley-function-call-leaderboard subproject that handles function-call evaluation using AST analysis, multi-turn evaluations, and model handlers (both for OSS and proprietary models).
- Training Pipeline: Represent the training modules used for fine-tuning models (SFT) and Schema Reinforcement Learning (SRL). Include the train directory (with SFT training scripts and data) and the included submodule (modified PRIME).
- Utility & Scripts Layer: Show the support provided by various bash scripts in the “scripts” folder (init_env.sh, train_sft.sh, train_srl.sh, test scripts) which orchestrate the overall workflow.

##### Relationships and Data Flow
- Data flows from the downloaded schema files (placed in the proper directories) into the SyntaxParser. The SyntaxParser transforms raw data according to supported syntaxes.
- Preprocessed data is then fed into the “bench” components which organize different testing benchmarks and evaluation routines.
- The training pipelines (SFT and SRL) use this data to fine-tune the language models. The “train” folder houses the training configurations and scripts which invoke sub-processes (e.g., via modified PRIME) for Schema RL.
- Trained models are then hooked up to the evaluation system, where CodeLinker (an external inference tool) is configured via a private configuration file and used in shell scripts (test_schemabench.sh) to run the evaluations.
- The berkeley-function-call-leaderboard module, which is more elaborate and modular, provides detailed performance evaluation with multiple metrics. This module is connected with both the training outputs (models) and the evaluation inputs (test data).

##### Architectural Patterns & Design Principles
- Modular design: Each top-level module (SyntaxParser, bench, berkeley-function-call-leaderboard, train, scripts) is clearly separated, allowing independent development and testing.
- Extensibility: The design supports multiple syntaxes (one parser per file type) and additional model handlers (OSS and proprietary) that can be plugged into the evaluation framework.
- Pipeline orchestration: The training and evaluation components are orchestrated using bash scripts, ensuring reproducible workflows.
- Integration of External Tools: The use of submodules (for the modified PRIME) and integration with CodeLinker for inference demonstrate both internal and external dependency flows.

##### Relevant Technologies & Frameworks
- Language: Python (with Python 3.11 specific environment setup).
- Scripting & Orchestration: Bash scripts are used to initialize environment, start training, and run evaluation.
- External Integrations: CodeLinker supports OpenAI compatible servers.
- Submodule: Modified version of PRIME for SRL is included as a Git submodule for model training.
- Data Formats: A wide range of schema formats (CSV, HTML, JSON, Markdown, etc.) are supported by the parser, highlighting a versatile parsing framework.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data & Schema Layer
    subgraph "Data & Schema Layer"
        DS1["Raw Schemas (schemabench/bench)"]:::data
        DS2["Raw Schemas (berkeley-function-call-leaderboard/data)"]:::data
    end

    %% Data Preprocessing & Parsing
    subgraph "Data Preprocessing & Parsing"
        DP["SyntaxParser Module"]:::processing
    end

    %% Benchmark & Evaluation Modules
    subgraph "Benchmark & Evaluation Modules"
        BE1["Benchmark Routines"]:::evaluation
        BE2["Function-Call Evaluation"]:::evaluation
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        TP1["SFT Training"]:::training
        TP2["SRL & PRIME Integration"]:::training
    end

    %% Inference Setup & Integration
    subgraph "Inference Setup & Integration"
        IS["CodeLinker Integration"]:::inference
    end

    %% Utility & Scripts Layer
    subgraph "Utility & Scripts Layer"
        US["Bash Scripts"]:::scripts
    end

    %% Data Flow Connections
    DS1 -->|"raw data"| DP
    DS2 -->|"raw data"| DP
    DP -->|"processed schemas"| BE1
    DP -->|"processed schemas"| TP1
    DP -->|"processed schemas"| TP2
    TP1 -->|"trained models"| BE1
    TP2 -->|"trained models"| BE1
    TP1 -->|"model outputs"| BE2
    TP2 -->|"model outputs"| BE2
    BE1 -->|"evaluation data"| BE2
    BE1 -->|"evaluation"| IS
    IS -->|"inference results"| US

    %% Click Events for Component Mappings
    click DS1 "https://github.com/thunlp/schemareinforcementlearning/tree/main/schemabench/bench"
    click DS2 "https://github.com/thunlp/schemareinforcementlearning/tree/main/schemabench/berkeley-function-call-leaderboard/data"
    click DP "https://github.com/thunlp/schemareinforcementlearning/tree/main/schemabench/SyntaxParser"
    click BE1 "https://github.com/thunlp/schemareinforcementlearning/tree/main/schemabench/bench"
    click BE2 "https://github.com/thunlp/schemareinforcementlearning/tree/main/schemabench/berkeley-function-call-leaderboard"
    click TP1 "https://github.com/thunlp/schemareinforcementlearning/tree/main/train/sft"
    click TP2 "https://github.com/thunlp/schemareinforcementlearning/tree/main/train/PRIME"
    click IS "https://github.com/thunlp/schemareinforcementlearning/blob/main/private_example.toml"
    click US "https://github.com/thunlp/schemareinforcementlearning/blob/main/scripts/test_schemabench.sh"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef processing fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef evaluation fill:#D8BFD8,stroke:#000,stroke-width:2px;
    classDef training fill:#FFA07A,stroke:#000,stroke-width:2px;
    classDef inference fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef scripts fill:#D3D3D3,stroke:#000,stroke-width:2px;
```
