How agent-infra/sandbox works
1. What this repo is
This is a monorepo for an “all-in-one” sandbox product: a Dockerized runtime that exposes multiple agent-facing capabilities, plus language SDKs, docs, examples, and an evaluation harness. The actual sandbox service is consumed as a container image; the repo mostly contains the client surfaces, packaging, and surrounding tooling.
2. Primary runtime boundary
The deployable unit is a single container listening on port 8080. From that one runtime, users reach:
- HTTP APIs under /v1
- VNC browser access
- VSCode Server (/code-server/)
- MCP endpoints (/mcp)
The README and compose file emphasize a shared filesystem and a unified execution environment inside the container.
3. Core service domains
The API is organized into capability-specific resource groups, not a single generic command endpoint. The main domains are:
- sandbox context / environment info
- shell sessions and process control
- file operations
- browser automation
- code execution
- Jupyter and Node.js runtimes
- proxy management
- MCP tool access
- skills registration/content
- browser state, cookies, network, tabs, captcha, page actions
4. Shared workspace as the main data plane
A key architectural feature is the single shared workspace inside the container. Data produced in one subsystem is immediately available to others:
- browser downloads can be read by file APIs
- shell commands can manipulate the same files
- Jupyter/Node.js can write artifacts into the workspace
This shared state is the main integration boundary across the whole product.
5. Browser subsystem
Browser support is broader than “open a page”: it includes page actions, tabs, cookies, network inspection/routing, screenshots, recording, captcha handling, and state persistence. The README also shows CDP/VNC usage, so browser automation can be driven both programmatically and visually. In diagrams, this is a major user-facing lane separate from shell/file.
6. Shell and terminal subsystem
Shell support is session-based and includes create/list/exec/write/view/wait/kill/cleanup flows. The README calls out a web terminal, which implies interactive process sessions rather than only one-shot command execution. This subsystem is the primary path for command-line work inside the sandbox.
7. Code execution runtimes
Python and Node.js are modeled as dedicated execution services, not just shell commands:
- Jupyter for notebook-style Python execution and session management
- Node.js for JS execution and session lifecycle
These are separate from shell, which matters for architecture because they imply isolated runtime contexts with their own session state and outputs.
8. File and content tooling
The file API is a first-class subsystem with read/write/list/search/grep/glob/find/replace/upload/download operations. The presence of search, replace-in-file, and str-replace-editor endpoints suggests the file layer is designed for agent editing workflows, not just basic storage access.
9. MCP and skills as the agent integration layer
MCP is positioned as the tool-execution boundary for agents, with servers/tools for browser, file, shell, and markdown conversion. Skills management is a separate capability for registering, listing, retrieving, and clearing reusable skills/metadata. In an architecture diagram, these sit between the core sandbox services and external agent frameworks.
10. Proxy and connectivity control
There is an explicit proxy subsystem for mappings, excludes, and diagnosis. This is important because the sandbox is meant to access external websites and services while staying within controlled networking rules. The README’s environment variables for proxy and DNS-over-HTTPS reinforce that outbound connectivity is a managed concern.
11. SDK surfaces
The repo ships generated client SDKs for JavaScript/TypeScript and Python, and references a Go SDK as well. The JS SDK has a shared fetch/core layer and per-resource clients; the Python SDK uses generated client wrappers and typed models. These SDKs are the main consumer-facing boundary to the HTTP API.
12. Examples and integration patterns
The examples/ tree is not core runtime code; it is a set of isolated sample projects showing how to combine the sandbox with external tools like Playwright, Browser Use, OpenAI, LangGraph, and S3. These examples are useful for understanding intended data flows, especially browser-to-file-to-code workflows.
13. Documentation and API contract
The website/ package is a full docs site in English and Chinese, with guides, examples, API pages, and the published OpenAPI spec in public/v1/openapi.json. This means the repo treats the API contract as a first-class artifact, and the docs site is part of the product boundary.
14. Evaluation and release pipeline
The evaluation/ directory contains benchmark datasets, an agent loop, parsing, and captured results, so there is a separate validation pipeline for sandbox capabilities. GitHub Actions build/publish the container image and SDKs, indicating the repo’s outputs are split into runtime image, SDK packages, and documentation releases.
Main entry points: README.md HTTP API · mcp MCP tools · file File tools · browserPage Browser automation
Components 22
Sandbox runtime
Deployable container and its exposed capability domains.
- Sandbox image · Single Dockerized service on port 8080 exposing HTTP, VNC, code-server, and MCP.
README.md - HTTP API · Primary /v1 control plane for sandbox capabilities.
README.md - Shared workspace · Common filesystem used by shell, browser downloads, code runtimes, and file APIs.
README.md - Shell sessions · Session-based terminal execution, output, wait, kill, and cleanup flows.
sdk/js/src/api/resources/shell - File tools · Read/write/list/search/replace/upload/download operations over the shared filesystem.
sdk/js/src/api/resources/file - Browser automation · Page actions, tabs, screenshots, recording, and browser state interactions.
sdk/js/src/api/resources/browserPage - Network control · Proxying, routing, headers, request capture, cookies, and HAR export for browser traffic.
sdk/js/src/api/resources/browserNetwork - Browser state · Persist and restore browser session state across agent runs.
sdk/js/src/api/resources/browserState - Tabs · Create, activate, list, and close browser tabs.
sdk/js/src/api/resources/browserTabs - Code execution · Dedicated code execution boundary distinct from shell.
sdk/js/src/api/resources/code - Jupyter · Notebook-style Python sessions, execution, and lifecycle management.
sdk/js/src/api/resources/jupyter - Node.js · Managed JavaScript sessions with execute and session lifecycle APIs.
sdk/js/src/api/resources/nodejs - Proxy control · Outbound mapping, exclusions, and diagnosis for managed connectivity.
sdk/js/src/api/resources/proxy - MCP tools · Tool execution boundary for browser, file, shell, and markdown conversion tools.
sdk/js/src/api/resources/mcp - Skills · Register, list, retrieve, and clear reusable skill metadata/content.
sdk/js/src/api/resources/skills
SDK clients
Generated client surfaces for the HTTP API.
- JS SDK · Generated TypeScript/JavaScript client with core fetch layer and per-resource wrappers.
sdk/js - Python SDK · Generated Python client package with typed models and resource clients.
sdk/python - Go SDK · Referenced Go client surface for the same API boundary.
sdk/go
Product surface
Documentation, examples, and evaluation tooling around the runtime.
- Docs site · Bilingual documentation site and published OpenAPI contract.
website/docs - Examples · Isolated integration examples for browser, code, agents, storage, and providers.
examples - Evaluation · Datasets, agent loop, parsing, and captured benchmark results.
evaluation - GitHub Actions · Build, publish, and SDK CI workflows for runtime image and client releases.
.github/workflows
Connections 33
- Sandbox image → HTTP API: exposes · The container is the single runtime boundary for the API and interactive surfaces.
- Sandbox image → Shared workspace: hosts · All major capabilities share one filesystem inside the container.
- HTTP API → Shell sessions: routes to · Shell endpoints manage persistent terminal/process sessions.
- HTTP API → File tools: routes to · File endpoints operate on the shared workspace.
- HTTP API → Browser automation: routes to · Browser endpoints drive page-level automation and stateful interactions.
- HTTP API → Network control: routes to · Network endpoints manage browser traffic controls and captures.
- HTTP API → Browser state: routes to · State endpoints persist and restore browser context.
- HTTP API → Tabs: routes to · Tab endpoints manage browser windows and page lifecycle.
- HTTP API → Code execution: routes to · Dedicated code execution boundary for programmatic workloads.
- HTTP API → Jupyter: routes to · Python notebook sessions are managed separately from shell.
- HTTP API → Node.js: routes to · JavaScript sessions are managed separately from shell.
- HTTP API → Proxy control: routes to · Proxy endpoints control external connectivity.
- HTTP API → MCP tools: routes to · MCP endpoints expose sandbox tools to agents.
- HTTP API → Skills: routes to · Skills endpoints manage reusable agent content and metadata.
- Shell sessions → Shared workspace: reads/writes · Terminal sessions manipulate the same filesystem as other subsystems.
- File tools → Shared workspace: reads/writes · File tools are the direct filesystem interface.
- Browser automation → Shared workspace: exports to · Browser downloads and artifacts land in shared storage.
- Jupyter → Shared workspace: writes artifacts · Python notebooks can create files visible to shell and file tools.
- Node.js → Shared workspace: writes artifacts · Node.js execution can persist outputs into the shared filesystem.
- MCP tools → Shell sessions: wraps · Tool boundary can invoke shell capabilities for agents.
- MCP tools → File tools: wraps · Tool boundary can invoke file capabilities for agents.
- MCP tools → Browser automation: wraps · Tool boundary can invoke browser capabilities for agents.
- JS SDK → HTTP API: consumes · Generated JS/TS client is the main typed consumer of the HTTP API.
- Python SDK → HTTP API: consumes · Generated Python client is the main typed consumer of the HTTP API.
- Go SDK → HTTP API: consumes · Go SDK mirrors the same API surface.
- Docs site → HTTP API: documents · Docs site publishes guides and the OpenAPI contract.
- Examples → JS SDK: shows usage · Examples demonstrate API usage patterns through client SDKs.
- Examples → Browser automation: demonstrates · Examples cover browser automation flows.
- Examples → File tools: demonstrates · Examples cover file-centric workflows.
- Evaluation → Sandbox image: benchmarks · Evaluation harness measures behavior of the sandbox image.
- GitHub Actions → Sandbox image: builds · Workflows publish the container image.
- GitHub Actions → JS SDK: publishes · Workflows test and publish the JS SDK.
- GitHub Actions → Python SDK: publishes · Workflows test and publish the Python SDK.