How alexzhang13/rlm works
Purpose and Public Entry Points
This is a Python RLM library: it wraps API/local language models in a recursive execution harness, allowing a root model to inspect offloaded context in a Python REPL and invoke LM or nested RLM calls. Public imports originate at rlm/__init__.py; the principal orchestration API is rlm/core/rlm.py. Examples such as examples/quickstart.py and examples/rlm_query_batched_example.py demonstrate application integration.
RLM Orchestrator and Completion Lifecycle
RLM constructs a model backend plus execution environment, then drives completion() iterations. It supplies the prompt/context/history to the environment, accepts generated code, executes it, feeds observations back to the root LM, and terminates with an RLMChatCompletion-style result and optional trajectory metadata.
• rlm/core/rlm.py
• rlm/core/types.py
• rlm/core/lm_handler.py
LM Abstraction and Recursive Dispatch
The LM handler mediates root and subcalls, including batched calls and concurrency/depth constraints. REPL-visible functions such as single/batched LM queries and recursive RLM queries route from generated code back through host-side orchestration, a necessary boundary for isolated environments. Provider adapters normalize provider-specific API behavior behind the base interface.
• rlm/core/comms_utils.py
• rlm/clients/base_lm.py
• rlm/clients/openai.py
Supported external model services include OpenAI/Azure OpenAI, Anthropic, Gemini, and Portkey/OpenRouter-style routing; local vLLM is accessed through the OpenAI-compatible client.
• rlm/clients/anthropic.py
• rlm/clients/gemini.py
• rlm/clients/portkey.py
REPL Execution and Isolation Boundary
Environment implementations execute model-authored Python against context and tools. LocalREPL is the default same-process exec environment; IPython can run in process or a kernel subprocess. Persistent sessions retain versioned context/history between completions, while compaction summarizes running history. This is the main trust boundary: local execution shares host dependencies and is unsuitable for untrusted production inputs.
• rlm/environments/base_env.py
• rlm/environments/local_repl.py
• rlm/environments/ipython_repl.py
Container/cloud implementations isolate code but proxy recursive LM requests to the host. Docker requires a Docker runtime; Modal, Prime, Daytona, and E2B rely on their corresponding sandbox services/SDKs.
• rlm/environments/docker_repl.py
• rlm/environments/modal_repl.py
• rlm/environments/prime_repl.py
Prompts, Parsing, and Execution Contracts
Core utility modules define system prompts, parse model outputs into executable actions/results, count or manage tokens, and centralize domain exceptions. These form the contract between model text generation, REPL execution, and the iterative controller.
• rlm/utils/prompts.py
• rlm/utils/parsing.py
• rlm/utils/token_utils.py
Observability and Trajectory Visualization
The logger records configuration, root iterations, code execution, and nested calls as in-memory metadata and optionally JSONL files. A separate Next.js visualizer ingests these logs client-side and presents execution, trajectory, iteration, and aggregate views.
• rlm/logger/rlm_logger.py
• visualizer/src/lib/parse-logs.ts
• visualizer/src/components/Dashboard.tsx
Training Subproject
training/ is an intentionally separate Python package integrating RLM execution with verifiers and Prime Intellect prime-rl. It exposes an RLM-backed task environment, uses subprocess-isolated local REPL execution, and defines workers, proxying, scoring rubrics, and configurable training jobs. OOLONG is the worked long-context QA environment.
• training/src/rlm_train/env.py
• training/src/rlm_train/worker.py
• training/environments/oolong/oolong/env.py
Documentation and Delivery
The distributable Python package is defined by pyproject.toml and targets Python 3.11+. Documentation is a separate Next.js site, while GitHub Actions publish the package and deploy/check docs. The docs and visualizer are supporting Node.js applications, not runtime dependencies of inference.
• pyproject.toml
• docs/src/app/page.tsx
• .github/workflows/publish.yml
Main entry points: rlm.py RLM orchestrator · base_env.py Environment interface · base_lm.py Base LM interface · lm_handler.py LM handler
Components 22
RLM Runtime
Iterative recursive model execution.
- Public API
rlm/__init__.py - RLM orchestrator · Builds runtime and drives completion iterations.
rlm/core/rlm.py - Completion types
rlm/core/types.py - LM handler · Enforces batching, concurrency, and recursion depth.
rlm/core/lm_handler.py - Prompt & parsing contract · Turns model output into executable actions and results.
rlm/utils/parsing.py - Token management
rlm/utils/token_utils.py - REPL-host bridge · Routes REPL-visible LM and nested-RLM calls to the host.
rlm/core/comms_utils.py
Execution Environments
Model-authored code execution boundary.
- Environment interface
rlm/environments/base_env.py - Local REPL · Default exec environment; not safe for untrusted code.
rlm/environments/local_repl.py - IPython REPL · Runs in-process or in a kernel subprocess.
rlm/environments/ipython_repl.py - Container & cloud REPLs · Proxy recursive LM requests back to the host.
rlm/environments/docker_repl.py
Model Providers
Normalized external model access.
- Base LM interface
rlm/clients/base_lm.py - OpenAI-compatible client · Also supports Azure and local vLLM endpoints.
rlm/clients/openai.py - Other provider adapters · Anthropic, Gemini, and Portkey/OpenRouter routing.
rlm/clients/anthropic.py
Observability & Tools
- RLM logger · Captures iterations, execution, and nested calls in memory or JSONL.
rlm/logger/rlm_logger.py - Trajectory visualizer · Client-side dashboard for logged trajectories.
visualizer/src/components/Dashboard.tsx - Package distribution · Targets Python 3.11+.
pyproject.toml - Documentation site · Separate from inference runtime.
docs/src/app/page.tsx - Publish workflow · Publishes package and deploys/checks docs.
.github/workflows/publish.yml
Training Package
Separate RLM training integration.
- RLM training environment · Integrates RLM execution with verifiers and prime-rl.
training/src/rlm_train/env.py - Training workers
training/src/rlm_train/worker.py - OOLONG environment
training/environments/oolong/oolong/env.py
Connections 22
- Public API → RLM orchestrator: exports
- RLM orchestrator → LM handler: dispatches calls
- RLM orchestrator → Environment interface: runs generated code
- RLM orchestrator → Prompt & parsing contract: builds and parses loop
- RLM orchestrator → Token management: manages context
- RLM orchestrator → Completion types: returns
- RLM orchestrator → RLM logger: records trajectory
- Environment interface → Local REPL: implemented by
- Environment interface → IPython REPL: implemented by
- Environment interface → Container & cloud REPLs: implemented by
- Environment interface → REPL-host bridge: exposes callbacks
- REPL-host bridge → LM handler: host-side recursive calls
- LM handler → Base LM interface: uses
- Base LM interface → OpenAI-compatible client: implemented by
- Base LM interface → Other provider adapters: implemented by
- RLM logger → Trajectory visualizer: JSONL logs ingested by
- RLM training environment → RLM orchestrator: executes with
- Training workers → RLM training environment: runs jobs
- OOLONG environment → RLM training environment: task environment
- Package distribution → Public API: packages
- Publish workflow → Package distribution: publishes
- Publish workflow → Documentation site: deploys/checks