# openpipe/art architecture

> Architecture diagram of the GitHub repository openpipe/art, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openpipe/art
- Repository: https://github.com/openpipe/art

## Overview

##### Project Type and Purpose
- ART is an open-source Python library that implements a **client–server** RL training system.
- Purpose: enable any codebase to execute “agentic” workflows (inference rollouts) via an ART client, while offloading model inference and GRPO‐based training to a dedicated ART server.

##### File Structure
- **src/art/**: the core Python package.
- art/cli.py – entry-point for CLI
- art/backend.py & art/openai.py – client interfaces to different backends
- art/local/ & art/skypilot/ – standalone server backends (vLLM local, SkyPilot for remote GPU clusters)
- art/dev/ – developer utilities (custom training loops, engines)
- art/model.py, art/trajectories.py – in‐memory representations of models & episode data
- art/utils/ – shared utilities (benchmarking, logging, S3, retry, formatters)
- **examples/**: illustrative use cases (2048, Codenames, Tic Tac Toe, ART•E agent)
- **docs/**: documentation site (MDX pages, images, analytics)
- **assets/**: benchmark graphs
- **scripts/** & **skypilot-config.yaml**: ops scripts to launch clusters, kill GPU processes, publish releases

##### Extract Key Architecture Points from the README
- **Client**: OpenAI-compatible façade. Your code calls `art.Client().chat()`.
- **Server**: runs separately on GPU. Exposes inference & training loops:
- Inference: loads latest LoRA into vLLM, serves completions.
- Training: groups trajectories, applies GRPO, writes new LoRA, reloads it.
- LoRA checkpoints live in server’s local filesystem (or S3 via utilities).
- Backends: local (vLLM), SkyPilot (remote cluster), plus an OpenAI‐compatible shim.

##### Components
a. **User Code / Client**
- Your application or notebook
- ART Python Client (art.cli, art.backend)
b. **ART Server**
- Inference Module (vLLM + LoRA loader)
- Trajectory Store (in-memory or on-disk)
- Training Module (GRPO optimizer)
- Checkpoint Store (local FS or S3)
c. **Backends**
- Local Backend (art.local.service)
- SkyPilot Backend (art.skypilot.stop_server, utils)
- OpenAI Shim (art.openai)
d. **External Services**
- GPU machines (cloud or on-prem)
- HuggingFace/vLLM model hub
- S3 (optional, via art.utils.s3)
- OpenAI API (for shim)

##### Relationships & Data Flow
- **Inference Phase**
1. User Code → ART Client → ART Server (Inference Module) → vLLM → ART Server returns LLM responses → User Code
2. ART Server logs each message in a Trajectory
- **Rewarding & Training Phase**
1. Upon rollout completion, User Code assigns rewards, sends Trajectories → ART Server
2. ART Server blocks further inference, runs GRPO (Training Module) → new LoRA checkpoint
3. LoRA Checkpoint → loaded back into Inference Module → inference unblocks
- **Cross‐cutting**
- Checkpoints read/write (local FS or S3)
- Metrics & charts generation (art.utils.benchmarking)

##### Architectural Patterns & Principles
- **Separation of Concerns**: clear split between client (API/CLI) and server (inference+training).
- **Plugin Backends**: interchangeable local vs remote (SkyPilot) vs OpenAI shim.
- **Event‐driven Loop**: inference → trajectory collection → training → reload → repeat.
- **Modularity**: utilities folder under art/utils for cross‐cutting features (retry, logging, S3).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User Layer"
        direction TB
        UserCode["User Code / Examples"]:::user
    end

    %% Client Layer
    subgraph "Client (ART Python Client)" 
        direction TB
        ART_CLI["CLI (src/art/cli.py)"]:::core
        ART_BACKEND["API Façade (src/art/backend.py)"]:::core
        ART_OPENAI["OpenAI Shim (src/art/openai.py)"]:::core
    end

    %% Server Layer
    subgraph "ART Server"
        direction TB
        InferenceModule["Inference Module\n(vLLM + LoRA loader)"]:::core
        TrajectoryStore["Trajectory Store\n(in-memory)"]:::core
        TrainingModule["GRPO Training Module"]:::core
        CheckpointStore["Checkpoint Store\n(Local FS & S3)"]:::core
    end

    %% Server Backends
    subgraph "Server Backends"
        direction TB
        subgraph "Local Backend"
            direction TB
            LB_Service["Service Entrypoint\n(src/art/local/service.py)"]:::core
            LB_Interface["Backend Interface\n(src/art/local/backend.py)"]:::core
            LB_vLLM["vLLM Integration & LoRA\n(src/art/local/vllm.py)"]:::core
            LB_Checkpoints["Local Checkpoints\n(src/art/local/checkpoints.py)"]:::core
            LB_Tokenize["Tokenization Utils\n(src/art/local/tokenize.py)"]:::core
            LB_State["State Store\n(src/art/local/state.py)"]:::core
            LB_Train["Local GRPO Trainer\n(src/art/local/train.py)"]:::core
            LB_Pack["Packaging Artifacts\n(src/art/local/pack.py)"]:::core
        end
        subgraph "SkyPilot Backend"
            direction TB
            SP_Interface["Backend Interface\n(src/art/skypilot/backend.py)"]:::core
            SP_Stop["Cluster Shutdown\n(src/art/skypilot/stop_server.py)"]:::core
            SP_Utils["SkyPilot Utils\n(src/art/skypilot/utils.py)"]:::core
        end
    end

    %% External Services
    GPUHost["GPU Machines"]:::external
    HuggingFace["HuggingFace / vLLM Hub"]:::external
    S3["S3 Storage"]:::external
    OpenAIAPI["OpenAI API"]:::external

    %% Connections - User to Client
    UserCode --> ART_CLI
    UserCode --> ART_BACKEND
    ART_BACKEND --> ART_OPENAI
    ART_OPENAI -->|HTTPS| OpenAIAPI

    %% Client to Server
    ART_CLI -->|gRPC/HTTP| InferenceModule
    ART_BACKEND -->|gRPC/HTTP| InferenceModule

    %% Inference Phase
    InferenceModule -->|loads model| HuggingFace
    InferenceModule --> TrajectoryStore
    InferenceModule -->|serves completions| UserCode

    %% Training Phase
    UserCode -->|rewards & trajectories| TrajectoryStore
    TrajectoryStore --> TrainingModule
    TrainingModule --> CheckpointStore
    CheckpointStore --> InferenceModule

    %% Checkpoint Storage
    CheckpointStore -->|S3 API| S3
    CheckpointStore -->|local FS| LB_Checkpoints

    %% Backends usage
    InferenceModule --> LB_vLLM
    TrainingModule --> LB_Train
    LB_Service --> GPUHost
    SP_Interface --> GPUHost

    %% Styles
    classDef core fill:#BBDEFB,stroke:#0D47A1,stroke-width:1px
    classDef external fill:#C8E6C9,stroke:#1B5E20,stroke-width:1px
    classDef user fill:#FFE0B2,stroke:#E65100,stroke-width:1px

    %% Click Events
    click ART_CLI "https://github.com/openpipe/art/blob/main/src/art/cli.py"
    click ART_BACKEND "https://github.com/openpipe/art/blob/main/src/art/backend.py"
    click ART_OPENAI "https://github.com/openpipe/art/blob/main/src/art/openai.py"
    click LB_Service "https://github.com/openpipe/art/blob/main/src/art/local/service.py"
    click LB_Interface "https://github.com/openpipe/art/blob/main/src/art/local/backend.py"
    click LB_vLLM "https://github.com/openpipe/art/blob/main/src/art/local/vllm.py"
    click LB_Checkpoints "https://github.com/openpipe/art/blob/main/src/art/local/checkpoints.py"
    click LB_Tokenize "https://github.com/openpipe/art/blob/main/src/art/local/tokenize.py"
    click LB_State "https://github.com/openpipe/art/blob/main/src/art/local/state.py"
    click LB_Train "https://github.com/openpipe/art/blob/main/src/art/local/train.py"
    click LB_Pack "https://github.com/openpipe/art/blob/main/src/art/local/pack.py"
    click SP_Interface "https://github.com/openpipe/art/blob/main/src/art/skypilot/backend.py"
    click SP_Stop "https://github.com/openpipe/art/blob/main/src/art/skypilot/stop_server.py"
    click SP_Utils "https://github.com/openpipe/art/blob/main/src/art/skypilot/utils.py"
    click TrajectoryStore "https://github.com/openpipe/art/blob/main/src/art/trajectories.py"
    click TrainingModule "https://github.com/openpipe/art/blob/main/src/art/gather.py"
    click CheckpointStore "https://github.com/openpipe/art/blob/main/src/art/utils/s3.py"
```
