How accio-lab/dressage works
1. Overall shape
Dressage is a Python RL orchestration layer that sits between upstream slime training and agent execution environments. It adds two main capabilities: agent/tool execution through paddocks, and token-level trajectory recording through a shared proxy.
2. Top-level runtime boundaries
The repo splits into three major runtime domains:
- dressage/ for rollout, sandbox, proxy, reward, and training glue
- blackbox_server/ for the in-sandbox HTTP service used by blackbox agents
- slime/ as an upstream submodule that provides the training/rollout substrate Dressage hooks into
This is important for the diagram: Dressage extends slime rather than replacing it.
3. End-to-end control flow
The main flow is: slime rollout hooks invoke Dressage’s generate path, a paddock acquires or attaches to a sandbox session, the agent runs, and all model calls go through the Dressage proxy to inference. The proxy records token-wise evidence, then the session is finalized into trajectories and converted into slime samples for training.
4. Proxy as the central data recorder
dressage/proxy/ is the core observability and trajectory layer. It mediates LLM calls, records per-token metadata, supports generation interruption/resume via GenerationController, and handles tokenization strategies like TITO. It also owns session management and trajectory storage, which makes it the canonical source of rollout evidence.
5. Paddocks define interaction semantics
dressage/paddock/ abstracts “how the agent interacts with the world.”
- Whitebox paddock: Python-native tool loops where the agent directly calls tools.
- Blackbox paddock: the agent is an external HTTP process managed through blackbox_server.
This is the boundary between environment logic and sandbox placement.
6. Sandbox layer handles isolation and placement
dressage/sandbox/ provides the execution substrate, independent of agent type. There are two main providers:
- local bubblewrap pool under local/bwrap/, with manager/supervisor/slot lifecycle
- remote E2B under remote/e2b/
The sandbox layer exposes provider-neutral concepts like leases/specs, so paddocks can request a sandbox without caring whether it is local or remote.
7. BlackboxServer is the in-sandbox agent host
blackbox_server/ is a FastAPI service that runs inside a sandbox and hosts exactly one active agent session at a time. It uses an adapter pattern (adapters/) to support backends like opencode and openclaw, while proxying model calls back through the Dressage proxy. This server is the key boundary for blackbox agents: it decouples rollout control from the concrete agent process.
8. Rollout orchestration and async modes
dressage/rollout/ contains the rollout engines and slime integration hooks. It supports sync, fully async, and partial async execution, plus multi-segment handling for trajectories that split due to compaction or schema changes. The generate/ subpackage contains the two execution entry points: blackbox dispatch and whitebox agent scaffolding.
9. Training/data conversion pipeline
dressage/training/ and dressage/rollout/artifacts/ turn recorded trajectories into training-ready samples. This layer handles reward post-processing, pause/resume semantics, and conversion of one trajectory into multiple samples when segments exist. The training path is designed to preserve token-level metadata, segment lineage, and prompt-equal weighting before handing data back to slime.
10. Reward and recipe boundaries
dressage/reward/ provides a registry and helpers for task-specific reward logic, while dressage/recipes/ contains end-to-end example agents for ALFWorld and HotpotQA. These recipes show how a concrete task plugs into the same paddock/proxy/sandbox pipeline without changing the core runtime architecture.
11. Configuration and runtime plumbing
dressage/config/ and dressage/runtime/ centralize environment-driven defaults, path resolution, and service naming. In practice, these modules glue together sandbox endpoints, proxy locations, model/runtime paths, and rollout behavior so that the same architecture can run locally, in containers, or against remote sandboxes.
12. What to emphasize in an architecture diagram
The most important diagram boundaries are:
- slime training loop vs Dressage rollout extensions
- paddock logic vs sandbox isolation
- in-sandbox blackbox agent vs external proxy/inference service
- proxy recording vs training conversion
The critical data path is not “agent to model” directly; it is agent → proxy → inference → proxy → trajectory store → sample conversion → training.
Main entry points: proxy Proxy · paddock.py Blackbox · sandbox Sandbox · rollout Rollout
Components 21
slime
Upstream training and rollout substrate
- Slime · Upstream rollout/training base that Dressage extends
slime
dressage
Core orchestration, proxy, sandbox, and training layers
- Rollout · Sync, async, and multi-segment rollout engines and slime hooks
dressage/rollout - Blackbox gen · Rollout entrypoint for launching blackbox agent sessions
dressage/rollout/generate/blackbox_dispatch.py - Whitebox gen · Rollout entrypoint for native agent/tool loops
dressage/rollout/generate/whitebox_agent.py - Paddock · Abstracts how an agent interacts with the world
dressage/paddock - Whitebox · Python-native tool execution inside Dressage
dressage/paddock/whitebox/paddock.py - Blackbox · Manages external agent sessions through blackbox_server
dressage/paddock/blackbox/paddock.py - Sandbox · Provider-neutral sandbox leasing and placement
dressage/sandbox - Bwrap pool · Local bubblewrap manager, supervisor, and slot lifecycle
dressage/sandbox/local/bwrap - E2B · Remote sandbox provider
dressage/sandbox/remote/e2b/provider.py - Proxy · Central LLM mediation, token recording, session and trajectory storage
dressage/proxy - Gen controller · Coordinates generation interruption, resumption, and last-step handling
dressage/proxy/generation_controller.py - Trajectory store · Persists token-level rollout evidence and session trajectories
dressage/proxy/trajectory_store.py - Training · Post-processes trajectories into training-ready samples
dressage/training - Reward · Reward registry and helpers for task-specific scoring
dressage/reward - Recipes · End-to-end task integrations such as ALFWorld and HotpotQA
dressage/recipes - Config/runtime · Environment defaults, path resolution, and service naming
dressage/config
blackbox server
In-sandbox HTTP host for blackbox agents
- API · HTTP endpoints for health, rollout, and sessions
blackbox_server/api - Adapters · Backend adapters for opencode, openclaw, and other agents
blackbox_server/adapters - LLM proxy · Forwards model calls back through Dressage proxy from inside the sandbox
blackbox_server/proxy/rollout_llm_proxy.py - Session store · Tracks the single active blackbox session
blackbox_server/store/session_store.py
Connections 23
- Slime → Rollout: rollout hooks · Dressage extends slime's training/rollout substrate
- Rollout → Blackbox gen: dispatch · Rollout selects the blackbox generation path
- Rollout → Whitebox gen: dispatch · Rollout selects the whitebox generation path
- Blackbox gen → Blackbox: session launch · Blackbox generation acquires or attaches to a sandboxed agent session
- Whitebox gen → Whitebox: tool loop · Whitebox generation scaffolds native agent/tool execution
- Paddock → Sandbox: lease sandbox · Paddocks request provider-neutral sandbox placement
- Blackbox → API: hosted in · Blackbox paddock talks to the in-sandbox HTTP agent server
- Whitebox → Proxy: model calls · Whitebox executions route LLM requests through the shared proxy
- Blackbox → Proxy: model calls · Blackbox agent sessions route LLM requests through the shared proxy
- Proxy → Gen controller: control · Proxy manages interruption, resume, and last-step behavior
- Proxy → Trajectory store: record · Proxy persists token-level evidence and session trajectories
- Trajectory store → Training: samples · Recorded trajectories are converted into training-ready samples
- Reward → Training: post-process · Reward outputs feed training-time reward processing
- Sandbox → Bwrap pool: local provider · Local bubblewrap pool supplies sandbox instances
- Sandbox → E2B: remote provider · Remote E2B supplies sandbox instances
- Blackbox → Session store: lifecycle · Blackbox paddock manages the active session lifecycle
- API → Adapters: dispatch · API routes agent requests to the selected backend adapter
- API → LLM proxy: proxy out · In-sandbox server forwards model calls back to Dressage proxy
- Session store → LLM proxy: session bound · The active blackbox session is tied to proxy-mediated rollouts
- Recipes → Whitebox: examples · Recipes demonstrate concrete whitebox agent integrations
- Recipes → Reward: task logic · Recipes provide task-specific reward implementations
- Config/runtime → Sandbox: defaults · Configuration resolves sandbox endpoints and provider settings
- Config/runtime → Proxy: defaults · Configuration resolves proxy locations and runtime paths