How thudm/slime works
slime is an LLM post-training framework centered on a shared RL loop: prompts or custom generation workflows produce rollouts, reward and verifier results are attached, samples enter the rollout data path, and Megatron trains on them before updated weights are synchronized to SGLang. The README also documents agent/environment interaction, asynchronous rollout, SGLang deployment options, and checkpoint/model conversion. Edges reflect README-described flow and sampled rollout/engine code; unsampled internals are represented only where the tree and README support the subsystem.
Main entry points: rollout.py Rollout coordinator · train_actor.py Training actor · actor.py Megatron backend · sglang_rollout.py SGLang generation
Components 23
Data generation
- Prompt sources
slime/rollout/data_source.py - Custom workflows
slime/rollout - Agent harnesses
slime/agent/harness - Rewards and verifiers
slime/rollout/rm_hub - Rollout coordinator
slime/ray/rollout.py - Rollout data buffer
slime_plugins/rollout_buffer/buffer.py
Rollout serving
- SGLang generation
slime/rollout/sglang_rollout.py - Engine groups
slime/backends/sglang_utils/engine_group.py - SGLang engine
slime/backends/sglang_utils/sglang_engine.py - External rollout
slime/backends/sglang_utils/external.py
Training
- Training actor
slime/ray/train_actor.py - Training entrypoint
train.py - Megatron backend
slime/backends/megatron_utils/actor.py - RL and OPD losses
slime/backends/megatron_utils/loss.py
Weights and models
- Checkpoint handling
slime/backends/megatron_utils/checkpoint.py - Weight synchronization
slime/backends/megatron_utils/update_weight - Model conversion
slime/backends/megatron_utils/hf_to_megatron
Runtime and observability
- Ray runtime
slime/ray/actor_group.py - Metrics and tracing
slime/observability
Other components
- Training operator
- Megatron
- SGLang
- Tools and environments
Connections 25
- Training operator → Training entrypoint: configures
- Prompt sources → Rollout coordinator: supplies prompts
- Custom workflows → Rollout coordinator: provides workflows
- Agent harnesses → Custom workflows: interacts with
- Custom workflows → Tools and environments: uses tools
- Rollout coordinator → SGLang generation: dispatches generation
- SGLang generation → Engine groups: requests inference
- Engine groups → SGLang engine: starts engines
- SGLang engine → SGLang: serves requests
- SGLang generation → Rewards and verifiers: computes rewards
- SGLang generation → Rollout data buffer: stores samples
- External rollout → Rollout coordinator: supplies rollouts
- Training entrypoint → Training actor: coordinates training
- Training actor → Rollout data buffer: reads samples
- Training actor → Megatron backend: runs training
- Megatron backend → Megatron: uses backend
- Megatron backend → RL and OPD losses: computes objectives
- Megatron backend → Checkpoint handling: saves state
- Training actor → Weight synchronization: synchronizes weights
- Weight synchronization → Engine groups: updates parameters
- Model conversion → Megatron backend: loads model weights
- Training entrypoint → Ray runtime: uses runtime
- Rollout coordinator → Ray runtime: uses runtime
- Rollout coordinator → Metrics and tracing: reports rollout data
- Training actor → Metrics and tracing: reports training data