# nick7nlp/fastcurl architecture

> Architecture diagram of the GitHub repository nick7nlp/fastcurl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nick7nlp/fastcurl
- Repository: https://github.com/nick7nlp/fastcurl

## Overview

##### Main Components
- Data Layer: Represent the training and test data stored as Parquet files (located under fastcurl/fastcurl/data/train and fastcurl/fastcurl/data/test). This layer is the source of problem-answer pairs compiled from various datasets.
- Core FastCuRL Module: This is in the fastcurl/fastcurl directory and encapsulates:
– global configuration (globals.py)
– utilities (utils.py)
– system prompts (system_prompts.py)
– reward computation modules (in rewards/ – includes math_reward and associated utilities)
- Training Orchestration & Scripts:
– The shell scripts in fastcurl/scripts/train (for each stage with different context lengths: 8K, 16K, 24K) initiate the training routines.
– The evaluation scripts under fastcurl/scripts/eval further rally the evaluation process.
- Distributed Training & Model Management:
– The fastcurl/verl directory is the heavily modified fork of “verl” which provides distributed orchestration. It contains:
▪ Model Definitions and Loaders (under verl/models – supporting multiple architectures such as LLaMA, Megatron, and Transformer-based models)
▪ Single controller / distributed communication modules (under verl/single_controller with submodules for Ray and Megatron)
▪ Trainer modules and configuration files (under verl/trainer which include YAML configuration for evaluation, generation, and PPO training)
▪ Utilities for distributed training, logging, debugging, and performance monitoring (under verl/utils)
▪ Third-party integrations with vLLM (under verl/third_party) which further facilitate efficient inference and weight loading.

##### Relationships and Interactions
- The training/evaluation scripts act as “entry points” that call into the core FastCuRL library for data handling and reward computation.
- The FastCuRL main module (fastcurl/fastcurl) provides the logic for reinforcement learning training, including processing prompts, applying reward signals, and managing curriculum stages.
- The “verl” component wraps the distributed model handling and training strategies:
– Training scripts invoke modules from verl/trainer.
– The distributed controller (from single_controller) schedules tasks across GPUs using Ray or Megatron, as seen in scripts and configuration files.
– Model checkpoint loading/saving and parallel executions are managed by components in verl/models and the third-party wrappers in vLLM directories.
- Data flows from the raw parquet datasets into the training pipeline, then through the model architecture where rewards (calculated by modules in fastcurl/fastcurl/rewards) are fed back to update model parameters iteratively.
- Configuration files (YAMLs) in verl/trainer/config help parameterize various training aspects, including evaluation metrics and generation settings.

##### Architectural Patterns and Design Principles
- Modularity: The system is highly modular with clear separation between data handling, reward computation, and distributed training/model management.
- Distributed / Parallel Training: Use of frameworks like Ray, Megatron, and vLLM that enable scaling training across multiple GPUs. This is evident in the distributed controller modules and scripts.
- Curriculum-based Reinforcement Learning: Training happens in multiple stages (evident from the four training stage scripts, each increasing the context length), making use of a progressive data presentation strategy.
- Reusability and extensibility: The separation into “fastcurl” (core functionality) and “verl” (extended distributed training and model components) suggests an architecture designed for reuse and further integration.

##### Elements
- Component Labels: Name each block (e.g., “Data Repository”, “FastCuRL Core”, “Reward Modules”, “Training Scripts”, “Distributed Controller (Ray/Megatron)”, “Model Repository”).
– From the “Data Repository” to "FastCuRL Core" (data ingestion).
– From "FastCuRL Core" to “Training Scripts” (training logic).
– From “Training Scripts” to “Distributed Training Modules” (orchestration across GPUs).
- Notation for Stages: Optionally, indicate the four training stages (Stage I: 8K, Stage II: 16K, Stage III: 24K, and Stage IV: 24K) near the training script section to capture the curriculum components.

##### Final Recommendations
- Be very detailed: Separate the project into as many logical components as possible to capture the nuances of its architecture.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Repository
    subgraph "Data Repository"
        DT["Train Data"]:::data
        DS["Test Data"]:::data
    end

    %% FastCuRL Core
    FC["FastCuRL Core"]:::core

    %% Reward Computation
    subgraph "Reward Computation"
        RM["Reward Modules"]:::reward
        MR["Math Reward"]:::rewardDetail
        MU["Math Utils"]:::rewardDetail
    end

    %% Training Orchestration
    subgraph "Training Orchestration"
        TS["Training Scripts<br/>(Stage I:8K,Stage II:16K,Stage III:24K,Stage IV:24K)"]:::train
        ES["Evaluation Scripts"]:::train
    end

    %% Distributed Training & Model Management
    subgraph "Distributed Training & Model Management"
        DC["Distributed Controller"]:::dist
        MRp["Model Repository"]:::dist
        TM["Trainer Modules"]:::dist
        DU["Distributed Utilities"]:::dist
        VI["vLLM Integration"]:::dist
    end

    %% Data flow arrows
    DT -->|"ingest"| FC
    DS -->|"ingest"| FC

    %% Core interactions
    FC -->|"applyReward"| RM
    FC -->|"initiateTraining"| TS
    RM -->|"rewardSignal"| TS
    ES -->|"feedback"| FC

    %% Training orchestration to distributed training
    TS -->|"orchestrate"| DC

    %% Distributed training internal flow
    DC -->|"loadModel"| MRp
    MRp -->|"config"| TM
    TM -->|"monitor"| DU
    DU -->|"integrate"| VI
    ES -->|"evaluate"| TM

    %% Click Events
    click DT "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/fastcurl/data/train"
    click DS "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/fastcurl/data/test"
    click FC "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/fastcurl"
    click RM "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/fastcurl/rewards"
    click MR "https://github.com/nick7nlp/fastcurl/blob/main/fastcurl/fastcurl/rewards/math_reward.py"
    click MU "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/fastcurl/rewards/math_utils"
    click TS "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/scripts/train"
    click ES "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/scripts/eval"
    click DC "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/verl/single_controller"
    click MRp "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/verl/models"
    click TM "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/verl/trainer"
    click DU "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/verl/utils"
    click VI "https://github.com/nick7nlp/fastcurl/tree/main/fastcurl/verl/third_party"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef core fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef reward fill:#FFEB99,stroke:#000,stroke-width:2px;
    classDef rewardDetail fill:#FFF68F,stroke:#000,stroke-width:2px;
    classDef train fill:#F4A460,stroke:#000,stroke-width:2px;
    classDef dist fill:#D8BFD8,stroke:#000,stroke-width:2px;
```
