# nvidia-nemo/prorl-agent-server architecture

> Architecture diagram of the GitHub repository nvidia-nemo/prorl-agent-server, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia-nemo/prorl-agent-server
- Repository: https://github.com/nvidia-nemo/prorl-agent-server

## Overview

1. **What the repo is**
   Polar is a rollout-as-a-service system for agentic RL. Its core architecture separates **task submission**, **distributed execution**, **trajectory construction**, and **evaluation** so training frameworks can consume rollouts without embedding agent harness logic directly.

2. **Top-level request flow**
   A client submits a rollout task through the **rollout service**. The rollout server acts as the central orchestrator, then assigns work to one or more **gateway nodes**. Gateways run the agent work asynchronously, collect outputs, and return structured rollout artifacts for later consumption.

3. **Rollout orchestration layer**
   `src/polar/rollout` contains the scheduling and control plane: balancing, pipeline management, timing, and the rollout server entrypoint. This layer is responsible for admission, dispatch, and coordinating many in-flight rollouts rather than executing the agent itself.

4. **Gateway execution layer**
   `src/polar/gateway` is the per-node runtime boundary. It includes node/session management, request proxying, dispatch, storage, and completion writing. This is where the system bridges between “task received” and “agent actually ran,” while keeping execution state isolated per node.

5. **Agent harness abstraction**
   `src/polar/agent` defines the harness contract and a set of presets for common agent styles such as shell-based agents, Claude Code, Codex, Gemini CLI, OpenHands, Qwen Code, and others. This makes agent execution backend-agnostic: Polar cares about the harness interface, not the specific agent implementation.

6. **Runtime isolation**
   `src/polar/runtime` abstracts how harnesses are launched, with support for **Docker** and **Apptainer**. Example directories include runtime Dockerfiles and image-build scripts, showing that Polar expects agent environments to be containerized and reproducible across different harnesses and benchmarks.

7. **Inference-server boundary**
   The repo explicitly supports both **SGLang** and **vLLM** via alternate topology files and patch scripts. Polar does not hardwire an inference engine; instead, gateways and agent harnesses communicate through provider-style HTTP boundaries so training can target whichever backend the harness supports.

8. **API transformation and proxying**
   `src/polar/gateway/transform` normalizes different model-provider request/response shapes, including OpenAI chat, OpenAI responses, Anthropic, Google, images, and reasoning metadata. This layer is important because it decouples agent code from a specific LLM API schema.

9. **Trajectory construction and evaluation**
   `src/polar/trajectory` turns raw execution traces into training-ready trajectories. Builders handle how records are merged or assembled per request, while evaluators score or validate outputs using strategies like session completion, test-on-output, SWE-bench harness integration, and other registered evaluators.

10. **Platform services and observability**
   `src/polar/platform` provides task/session/event APIs, topology exposure, an SSE fanout mechanism, and filesystem indexing for stored artifacts. This is the operational control plane that supports the dashboard and external tooling, separate from the execution pipeline itself.

11. **Trainer integration: Slime bridge**
   `src/slime_bridge` is the demo RL/training integration layer. It adapts Polar rollout outputs into training-time messages, reward computation, rollout acceptance logic, and post-processing, showing how Polar plugs into an external RL stack without owning the trainer.

12. **Examples as architectural blueprints**
   The `examples/` tree shows the expected deployment pattern end-to-end: each example has a task runner, a runtime image, and one or more topology files. The examples span a calculator smoke test, count-stars vision task, SWE-bench Verified evaluation, Tmax-15k, and a SWE-Gym/GRPO training path, demonstrating that the architecture is meant to generalize across benchmark, agentic coding, and training workloads.

13. **UI and operator boundary**
   `web/` is an optional React/Vite dashboard for monitoring tasks, sessions, traces, completions, topology, and reward curves. It sits on top of the platform APIs and SSE streams; it does not participate in rollout execution, only observation and comparison.

14. **Configuration model**
   The repo’s deployment unit is a shared **topology file**. The same topology drives `polar serve_rollout`, `polar serve_gateway`, `polar submit`, `polar status`, and the dashboard, which makes the system diagram naturally centered around a single configuration graph connecting orchestrator, gateways, runtimes, and inference backends.

15. **Key architectural boundary**
   The main design principle is that **agent logic, inference serving, runtime isolation, rollout orchestration, and evaluation are all separate layers**. That separation is what allows Polar to support many harnesses and trainers while keeping the rollout server itself neutral and scalable.

Main entry points: [`src/polar/gateway/node.py`](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/node.py) (Node runtime), [`src/polar/config/topology.py`](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/config/topology.py) (Topology), [`src/polar/rollout/server.py`](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/rollout/server.py) (Rollout server), [`src/polar/platform/server.py`](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/platform/server.py) (Platform server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control_plane["Control plane"]
  node_topology["Topology<br/>deployment config<br/>[topology.py]"]
  node_rollout_server["Rollout server<br/>orchestrator<br/>[server.py]"]
  node_rollout_manager["Manager<br/>rollout control<br/>[manager.py]"]
  node_rollout_balancer["Balancer<br/>scheduling<br/>[balancer.py]"]
  node_platform_server["Platform server<br/>ops api<br/>[server.py]"]
  node_platform_api_tasks["Task API<br/>task/session api<br/>[tasks.py]"]
  node_platform_sse(("SSE fanout<br/>event stream<br/>[sse_fanout.py]"))
end

subgraph group_execution["Execution"]
  node_gateway_server["Gateway server<br/>node runtime<br/>[server.py]"]
  node_gateway_node["Node runtime<br/>session manager<br/>[node.py]"]
  node_gateway_dispatcher["Dispatcher<br/>work dispatch<br/>[dispatcher.py]"]
  node_gateway_proxy["Proxy<br/>request bridge<br/>[proxy.py]"]
  node_api_transform["API transforms<br/>schema adapters<br/>[base.py]"]
  node_agent_factory["Agent factory<br/>harness registry<br/>[factory.py]"]
  node_runtime_factory["Runtime factory<br/>runtime launcher<br/>[factory.py]"]
  node_docker_runtime["Docker<br/>runtime backend<br/>[docker.py]"]
  node_apptainer_runtime["Apptainer<br/>runtime backend<br/>[apptainer.py]"]
end

subgraph group_trajectories["Trajectories"]
  node_traj_registry["Trajectory registry<br/>[registry.py]"]
  node_traj_builder["Builders<br/>trajectory assembly<br/>[base.py]"]
  node_traj_evaluator["Evaluators<br/>trajectory scoring<br/>[base.py]"]
end

subgraph group_integration["Training integration"]
  node_slime_adapter["Slime adapter<br/>rl bridge<br/>[adapter.py]"]
  node_slime_reward["Reward<br/>[reward.py]"]
end

subgraph group_examples_ui["Examples and UI"]
  node_example_topology["Example topologies<br/>deployment blueprints<br/>[topology.sgl.yaml]"]
  node_example_runner["Example runners<br/>task entrypoints<br/>[run.py]"]
  node_web_dashboard["Dashboard<br/>react ui<br/>[Dashboard.tsx]"]
  node_web_api_client["API client<br/>ui api<br/>[client.ts]"]
end

node_topology -->|"configure"| node_rollout_server
node_topology -->|"configure"| node_gateway_server
node_topology -->|"configure"| node_platform_server
node_rollout_server -->|"coordinate"| node_rollout_manager
node_rollout_manager -->|"schedule"| node_rollout_balancer
node_rollout_server -->|"dispatch"| node_gateway_server
node_gateway_server -->|"host"| node_gateway_node
node_gateway_node -->|"route"| node_gateway_dispatcher
node_gateway_dispatcher -->|"proxy"| node_gateway_proxy
node_gateway_proxy -->|"normalize"| node_api_transform
node_gateway_node -->|"build"| node_agent_factory
node_agent_factory -->|"launch"| node_runtime_factory
node_runtime_factory -->|"backend"| node_docker_runtime
node_runtime_factory -->|"backend"| node_apptainer_runtime
node_gateway_node -->|"emit"| node_traj_builder
node_traj_registry -->|"register"| node_traj_builder
node_traj_registry -->|"register"| node_traj_evaluator
node_traj_builder -->|"score"| node_traj_evaluator
node_slime_adapter -->|"consume"| node_rollout_server
node_slime_adapter -->|"reward"| node_slime_reward
node_platform_server -->|"serve"| node_platform_api_tasks
node_platform_server -->|"stream"| node_platform_sse
node_web_dashboard -->|"query"| node_web_api_client
node_web_api_client -->|"read"| node_platform_server
node_web_api_client -->|"subscribe"| node_platform_sse
node_example_topology -->|"drive"| node_example_runner
node_example_runner -->|"mirror"| node_topology

click node_topology "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/config/topology.py"
click node_rollout_server "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/rollout/server.py"
click node_rollout_manager "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/rollout/manager.py"
click node_rollout_balancer "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/rollout/balancer.py"
click node_platform_server "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/platform/server.py"
click node_platform_api_tasks "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/platform/api/tasks.py"
click node_platform_sse "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/platform/sse_fanout.py"
click node_gateway_server "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/server.py"
click node_gateway_node "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/node.py"
click node_gateway_dispatcher "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/dispatcher.py"
click node_gateway_proxy "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/proxy.py"
click node_api_transform "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/transform/base.py"
click node_agent_factory "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/agent/factory.py"
click node_runtime_factory "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/runtime/factory.py"
click node_docker_runtime "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/runtime/docker.py"
click node_apptainer_runtime "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/runtime/apptainer.py"
click node_traj_registry "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/trajectory/registry.py"
click node_traj_builder "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/trajectory/builder/base.py"
click node_traj_evaluator "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/trajectory/evaluator/base.py"
click node_slime_adapter "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/slime_bridge/adapter.py"
click node_slime_reward "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/slime_bridge/reward.py"
click node_example_topology "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/examples/calculator/topology.sgl.yaml"
click node_example_runner "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/examples/calculator/run.py"
click node_web_dashboard "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/web/src/routes/Dashboard.tsx"
click node_web_api_client "https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/web/src/api/client.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_topology,node_rollout_server,node_rollout_manager,node_rollout_balancer,node_platform_server,node_platform_api_tasks,node_platform_sse toneBlue
class node_gateway_server,node_gateway_node,node_gateway_dispatcher,node_gateway_proxy,node_api_transform,node_agent_factory,node_runtime_factory,node_docker_runtime,node_apptainer_runtime toneAmber
class node_traj_registry,node_traj_builder,node_traj_evaluator toneMint
class node_slime_adapter,node_slime_reward toneRose
class node_example_topology,node_example_runner,node_web_dashboard,node_web_api_client toneIndigo
```

## Components

### Control plane

Submission, orchestration, and platform APIs

- [Topology](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/config/topology.py): deployment config · Shared topology model that wires rollout, gateways, runtimes, and backends · `src/polar/config/topology.py`
- [Rollout server](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/rollout/server.py): orchestrator · Entry point for task admission, dispatch, and rollout coordination · `src/polar/rollout/server.py`
- [Manager](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/rollout/manager.py): rollout control · Coordinates in-flight rollouts and overall control flow · `src/polar/rollout/manager.py`
- [Balancer](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/rollout/balancer.py): scheduling · Assigns work across gateway nodes and load-balances rollout tasks · `src/polar/rollout/balancer.py`
- [Platform server](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/platform/server.py): ops api · Operational API surface for tasks, sessions, events, and topology · `src/polar/platform/server.py`
- [Task API](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/platform/api/tasks.py): task/session api · Task and session endpoints used by tools and UI · `src/polar/platform/api/tasks.py`
- [SSE fanout](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/platform/sse_fanout.py): event stream · Pushes live task and session updates to observers · `src/polar/platform/sse_fanout.py`

### Execution

Gateway runtime, agent harnesses, and model API adaptation

- [Gateway server](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/server.py): node runtime · Per-node execution boundary that receives assigned rollout work · `src/polar/gateway/server.py`
- [Node runtime](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/node.py): session manager · Maintains node and session state for isolated execution · `src/polar/gateway/node.py`
- [Dispatcher](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/dispatcher.py): work dispatch · Routes requests into the active agent execution path · `src/polar/gateway/dispatcher.py`
- [Proxy](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/proxy.py): request bridge · Bridges rollout traffic to provider-style model endpoints · `src/polar/gateway/proxy.py`
- [API transforms](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/gateway/transform/base.py): schema adapters · Normalizes OpenAI, Anthropic, Google, image, and reasoning payloads · `src/polar/gateway/transform/base.py`
- [Agent factory](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/agent/factory.py): harness registry · Builds agent harnesses from presets and execution models · `src/polar/agent/factory.py`
- [Runtime factory](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/runtime/factory.py): runtime launcher · Creates containerized launchers for harness execution · `src/polar/runtime/factory.py`
- [Docker](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/runtime/docker.py): runtime backend · Docker-based runtime isolation for harnesses · `src/polar/runtime/docker.py`
- [Apptainer](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/runtime/apptainer.py): runtime backend · Apptainer-based runtime isolation for harnesses · `src/polar/runtime/apptainer.py`

### Trajectories

Turn rollout artifacts into training-ready outputs and scores

- [Trajectory registry](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/trajectory/registry.py): registry · Registers builders and evaluators for trajectory assembly · `src/polar/trajectory/registry.py`
- [Builders](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/trajectory/builder/base.py): trajectory assembly · Combines raw records into training-ready trajectories · `src/polar/trajectory/builder/base.py`
- [Evaluators](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/polar/trajectory/evaluator/base.py): trajectory scoring · Scores or validates rollout outputs with pluggable strategies · `src/polar/trajectory/evaluator/base.py`

### Training integration

Bridge into external RL/training flows

- [Slime adapter](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/slime_bridge/adapter.py): rl bridge · Adapts Polar rollout outputs into trainer-facing messages and logic · `src/slime_bridge/adapter.py`
- [Reward](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/src/slime_bridge/reward.py): reward logic · Computes reward signals and post-processing for training · `src/slime_bridge/reward.py`

### Examples and UI

Deployment blueprints and the operator dashboard

- [Example topologies](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/examples/calculator/topology.sgl.yaml): deployment blueprints · Example rollout layouts for benchmarks and training stacks · `examples/calculator/topology.sgl.yaml`
- [Example runners](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/examples/calculator/run.py): task entrypoints · Scripts that submit benchmark or training tasks through Polar · `examples/calculator/run.py`
- [Dashboard](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/web/src/routes/Dashboard.tsx): react ui · Operator UI for monitoring tasks, sessions, traces, and rewards · `web/src/routes/Dashboard.tsx`
- [API client](https://github.com/nvidia-nemo/prorl-agent-server/blob/stable/web/src/api/client.ts): ui api · Frontend client for platform data and SSE streams · `web/src/api/client.ts`

## Connections

- Topology → Rollout server: configure · Topology drives rollout orchestration
- Topology → Gateway server: configure · Topology selects gateway nodes and backends
- Topology → Platform server: configure · Platform exposes the same deployment graph
- Rollout server → Manager: coordinate · Server delegates rollout lifecycle control
- Manager → Balancer: schedule · Manager assigns work through balancing logic
- Rollout server → Gateway server: dispatch · Rollout work is sent to gateway nodes
- Gateway server → Node runtime: host · Gateway keeps per-node state isolated
- Node runtime → Dispatcher: route · Node runtime sends requests into dispatch
- Dispatcher → Proxy: proxy · Dispatcher forwards provider-bound traffic
- Proxy → API transforms: normalize · Proxy normalizes request and response schemas
- Node runtime → Agent factory: build · Node instantiates the selected agent harness
- Agent factory → Runtime factory: launch · Harness execution is bound to a runtime launcher
- Runtime factory → Docker: backend · Docker is one supported isolated runtime
- Runtime factory → Apptainer: backend · Apptainer is the other supported isolated runtime
- Node runtime → Builders: emit · Execution outputs become trajectory records
- Trajectory registry → Builders: register · Registry provides builder implementations
- Trajectory registry → Evaluators: register · Registry provides evaluator implementations
- Builders → Evaluators: score · Built trajectories are passed into evaluation
- Slime adapter → Rollout server: consume · Training integration consumes rollout artifacts
- Slime adapter → Reward: reward · Adapter computes trainer-facing reward logic
- Platform server → Task API: serve · Platform exposes task and session APIs
- Platform server → SSE fanout: stream · Platform publishes live updates to observers
- Dashboard → API client: query · Dashboard uses the shared API client
- API client → Platform server: read · UI reads tasks, sessions, and topology from platform APIs
- API client → SSE fanout: subscribe · UI subscribes to live event streams
- Example topologies → Example runners: drive · Examples pair topology files with submission scripts
- Example runners → Topology: mirror · Examples demonstrate the same deployment model
