# thudm/slime architecture

> Architecture diagram of the GitHub repository thudm/slime, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thudm/slime
- Repository: https://github.com/thudm/slime

## Overview

slime is an LLM post-training framework centered on a shared RL loop: prompts or custom generation workflows produce rollouts, reward and verifier results are attached, samples enter the rollout data path, and Megatron trains on them before updated weights are synchronized to SGLang. The README also documents agent/environment interaction, asynchronous rollout, SGLang deployment options, and checkpoint/model conversion. Edges reflect README-described flow and sampled rollout/engine code; unsampled internals are represented only where the tree and README support the subsystem.

Main entry points: [`slime/ray/rollout.py`](https://github.com/thudm/slime/blob/main/slime/ray/rollout.py) (Rollout coordinator), [`slime/ray/train_actor.py`](https://github.com/thudm/slime/blob/main/slime/ray/train_actor.py) (Training actor), [`slime/backends/megatron_utils/actor.py`](https://github.com/thudm/slime/blob/main/slime/backends/megatron_utils/actor.py) (Megatron backend), [`slime/rollout/sglang_rollout.py`](https://github.com/thudm/slime/blob/main/slime/rollout/sglang_rollout.py) (SGLang generation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_generation["Data generation"]
  node_prompts["Prompt sources<br/>[data_source.py]"]
  node_customgen["Custom workflows"]
  node_agents["Agent harnesses"]
  node_rewards["Rewards and verifiers"]
  node_rollout["Rollout coordinator<br/>[rollout.py]"]
  node_buffer[("Rollout data buffer<br/>[buffer.py]")]
end

subgraph group_serving["Rollout serving"]
  node_sglangrollout["SGLang generation<br/>[sglang_rollout.py]"]
  node_engines["Engine groups<br/>[engine_group.py]"]
  node_engine["SGLang engine<br/>[sglang_engine.py]"]
  node_externalengine["External rollout<br/>[external.py]"]
end

subgraph group_training["Training"]
  node_trainer["Training actor<br/>[train_actor.py]"]
  node_trainloop["Training entrypoint<br/>[train.py]"]
  node_megatron["Megatron backend<br/>[actor.py]"]
  node_loss["RL and OPD losses<br/>[loss.py]"]
end

subgraph group_sync["Weights and models"]
  node_checkpoint["Checkpoint handling<br/>[checkpoint.py]"]
  node_weights["Weight synchronization"]
  node_conversion["Model conversion"]
end

subgraph group_infra["Runtime and observability"]
  node_ray["Ray runtime<br/>[actor_group.py]"]
  node_metrics["Metrics and tracing"]
end

node_user(("Training operator"))
node_megatron_service["Megatron"]
node_sglang_service["SGLang"]
node_environment["Tools and environments"]

node_user -->|"configures"| node_trainloop
node_prompts -->|"supplies prompts"| node_rollout
node_customgen -->|"provides workflows"| node_rollout
node_agents -->|"interacts with"| node_customgen
node_customgen -.->|"uses tools"| node_environment
node_rollout -->|"dispatches generation"| node_sglangrollout
node_sglangrollout -->|"requests inference"| node_engines
node_engines -->|"starts engines"| node_engine
node_engine -->|"serves requests"| node_sglang_service
node_sglangrollout -->|"computes rewards"| node_rewards
node_sglangrollout -->|"stores samples"| node_buffer
node_externalengine -.->|"supplies rollouts"| node_rollout
node_trainloop -->|"coordinates training"| node_trainer
node_trainer -->|"reads samples"| node_buffer
node_trainer -->|"runs training"| node_megatron
node_megatron -->|"uses backend"| node_megatron_service
node_megatron -->|"computes objectives"| node_loss
node_megatron -->|"saves state"| node_checkpoint
node_trainer -->|"synchronizes weights"| node_weights
node_weights -->|"updates parameters"| node_engines
node_conversion -.->|"loads model weights"| node_megatron
node_trainloop -->|"uses runtime"| node_ray
node_rollout -->|"uses runtime"| node_ray
node_rollout -->|"reports rollout data"| node_metrics
node_trainer -->|"reports training data"| node_metrics

click node_prompts "https://github.com/thudm/slime/blob/main/slime/rollout/data_source.py"
click node_customgen "https://github.com/thudm/slime/tree/main/slime/rollout"
click node_agents "https://github.com/thudm/slime/tree/main/slime/agent/harness"
click node_rewards "https://github.com/thudm/slime/tree/main/slime/rollout/rm_hub"
click node_rollout "https://github.com/thudm/slime/blob/main/slime/ray/rollout.py"
click node_sglangrollout "https://github.com/thudm/slime/blob/main/slime/rollout/sglang_rollout.py"
click node_engines "https://github.com/thudm/slime/blob/main/slime/backends/sglang_utils/engine_group.py"
click node_engine "https://github.com/thudm/slime/blob/main/slime/backends/sglang_utils/sglang_engine.py"
click node_externalengine "https://github.com/thudm/slime/blob/main/slime/backends/sglang_utils/external.py"
click node_buffer "https://github.com/thudm/slime/blob/main/slime_plugins/rollout_buffer/buffer.py"
click node_trainer "https://github.com/thudm/slime/blob/main/slime/ray/train_actor.py"
click node_trainloop "https://github.com/thudm/slime/blob/main/train.py"
click node_megatron "https://github.com/thudm/slime/blob/main/slime/backends/megatron_utils/actor.py"
click node_loss "https://github.com/thudm/slime/blob/main/slime/backends/megatron_utils/loss.py"
click node_checkpoint "https://github.com/thudm/slime/blob/main/slime/backends/megatron_utils/checkpoint.py"
click node_weights "https://github.com/thudm/slime/tree/main/slime/backends/megatron_utils/update_weight"
click node_conversion "https://github.com/thudm/slime/tree/main/slime/backends/megatron_utils/hf_to_megatron"
click node_ray "https://github.com/thudm/slime/blob/main/slime/ray/actor_group.py"
click node_metrics "https://github.com/thudm/slime/tree/main/slime/observability"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prompts,node_customgen,node_agents,node_rewards,node_rollout,node_buffer toneBlue
class node_sglangrollout,node_engines,node_engine,node_externalengine toneAmber
class node_trainer,node_trainloop,node_megatron,node_loss toneMint
class node_checkpoint,node_weights,node_conversion toneRose
class node_ray,node_metrics,node_user,node_megatron_service,node_sglang_service,node_environment toneIndigo
```

## Components

### Data generation

- [Prompt sources](https://github.com/thudm/slime/blob/main/slime/rollout/data_source.py): `slime/rollout/data_source.py`
- [Custom workflows](https://github.com/thudm/slime/tree/main/slime/rollout): `slime/rollout`
- [Agent harnesses](https://github.com/thudm/slime/tree/main/slime/agent/harness): `slime/agent/harness`
- [Rewards and verifiers](https://github.com/thudm/slime/tree/main/slime/rollout/rm_hub): `slime/rollout/rm_hub`
- [Rollout coordinator](https://github.com/thudm/slime/blob/main/slime/ray/rollout.py): `slime/ray/rollout.py`
- [Rollout data buffer](https://github.com/thudm/slime/blob/main/slime_plugins/rollout_buffer/buffer.py): `slime_plugins/rollout_buffer/buffer.py`

### Rollout serving

- [SGLang generation](https://github.com/thudm/slime/blob/main/slime/rollout/sglang_rollout.py): `slime/rollout/sglang_rollout.py`
- [Engine groups](https://github.com/thudm/slime/blob/main/slime/backends/sglang_utils/engine_group.py): `slime/backends/sglang_utils/engine_group.py`
- [SGLang engine](https://github.com/thudm/slime/blob/main/slime/backends/sglang_utils/sglang_engine.py): `slime/backends/sglang_utils/sglang_engine.py`
- [External rollout](https://github.com/thudm/slime/blob/main/slime/backends/sglang_utils/external.py): `slime/backends/sglang_utils/external.py`

### Training

- [Training actor](https://github.com/thudm/slime/blob/main/slime/ray/train_actor.py): `slime/ray/train_actor.py`
- [Training entrypoint](https://github.com/thudm/slime/blob/main/train.py): `train.py`
- [Megatron backend](https://github.com/thudm/slime/blob/main/slime/backends/megatron_utils/actor.py): `slime/backends/megatron_utils/actor.py`
- [RL and OPD losses](https://github.com/thudm/slime/blob/main/slime/backends/megatron_utils/loss.py): `slime/backends/megatron_utils/loss.py`

### Weights and models

- [Checkpoint handling](https://github.com/thudm/slime/blob/main/slime/backends/megatron_utils/checkpoint.py): `slime/backends/megatron_utils/checkpoint.py`
- [Weight synchronization](https://github.com/thudm/slime/tree/main/slime/backends/megatron_utils/update_weight): `slime/backends/megatron_utils/update_weight`
- [Model conversion](https://github.com/thudm/slime/tree/main/slime/backends/megatron_utils/hf_to_megatron): `slime/backends/megatron_utils/hf_to_megatron`

### Runtime and observability

- [Ray runtime](https://github.com/thudm/slime/blob/main/slime/ray/actor_group.py): `slime/ray/actor_group.py`
- [Metrics and tracing](https://github.com/thudm/slime/tree/main/slime/observability): `slime/observability`

### Other components

- **Training operator**
- **Megatron**
- **SGLang**
- **Tools and environments**

## Connections

- Training operator → Training entrypoint: configures
- Prompt sources → Rollout coordinator: supplies prompts
- Custom workflows → Rollout coordinator: provides workflows
- Agent harnesses → Custom workflows: interacts with
- Custom workflows → Tools and environments: uses tools
- Rollout coordinator → SGLang generation: dispatches generation
- SGLang generation → Engine groups: requests inference
- Engine groups → SGLang engine: starts engines
- SGLang engine → SGLang: serves requests
- SGLang generation → Rewards and verifiers: computes rewards
- SGLang generation → Rollout data buffer: stores samples
- External rollout → Rollout coordinator: supplies rollouts
- Training entrypoint → Training actor: coordinates training
- Training actor → Rollout data buffer: reads samples
- Training actor → Megatron backend: runs training
- Megatron backend → Megatron: uses backend
- Megatron backend → RL and OPD losses: computes objectives
- Megatron backend → Checkpoint handling: saves state
- Training actor → Weight synchronization: synchronizes weights
- Weight synchronization → Engine groups: updates parameters
- Model conversion → Megatron backend: loads model weights
- Training entrypoint → Ray runtime: uses runtime
- Rollout coordinator → Ray runtime: uses runtime
- Rollout coordinator → Metrics and tracing: reports rollout data
- Training actor → Metrics and tracing: reports training data