How alibaba/opensandbox works
OpenSandbox is a monorepo for a sandbox platform, and its architecture is organized around a clear separation between API contracts, control plane services, runtime agents, and client SDKs.
At a high level, the system has four major layers:
1. Client-facing SDKs
2. A central sandbox server
3. Runtime-side components that run inside or alongside sandboxes
4. A Kubernetes deployment path for large-scale sandbox orchestration
The repo also contains the API specs, example workloads, and end-to-end tests that validate the whole stack.
1) API contract layer
The specs/ directory is the architectural boundary that defines the platform’s public interfaces:
• sandbox-lifecycle.yml defines sandbox creation, listing, pause/resume, renew, delete, and endpoint discovery.
• execd-api.yaml defines runtime execution APIs for commands, files, health, metrics, and code interpreting.
• egress-api.yaml defines network policy and egress control APIs.
These specs are important because the SDKs, the Python server, and the runtime components are all built around them. In practice, this means the repo is not one monolith API; it is a system of separate services with explicit contracts.
2) Client SDK layer
The sdks/ directory contains multi-language client libraries for two different consumption patterns:
Sandbox SDK
Used to create and manage sandboxes, connect to them, and use services like:
• commands
• filesystem
• health
• metrics
• egress
• sandbox lifecycle management
This SDK exists in Python, JavaScript/TypeScript, Kotlin, and C#. The SDKs are structured similarly:
• a connection/config layer
• generated or OpenAPI-backed clients
• service adapters
• model classes
• optional synchronous wrappers
Code Interpreter SDK
A narrower SDK built on top of the sandbox and execd concepts. It focuses on code execution inside an already-provisioned sandbox and exposes code-oriented abstractions.
Architecturally, the SDKs are the outermost entry point: application code talks to the SDK, and the SDK talks to the server and runtime APIs over HTTP/SSE.
3) Central server layer
The server/ directory is the main control-plane service for non-Kubernetes deployment and for unified API access.
Runtime and role
The server is a Python FastAPI application. It exposes the platform’s user-facing HTTP APIs, and internally routes requests to different backends depending on the runtime mode:
• Docker-backed sandbox management
• Kubernetes-backed sandbox management
• runtime proxying to sandbox endpoints
• egress policy operations
• pool-related lifecycle operations
• devops/diagnostic endpoints
Main subsystems inside the server
• api/: HTTP route handlers for lifecycle, proxy, pool, schema, and devops
• services/: backend logic and orchestration
• services/k8s/: Kubernetes-specific providers, templates, informer integration, rate limiting, volume helpers, and pool handling
• services/docker.py and services/docker_diagnostics.py: Docker runtime support and diagnostics
• integrations/renew_intent/: a renewal flow that coordinates sandbox expiration extension when active traffic is observed
• middleware/: auth and request ID handling
• extensions/: extension loading and validation utilities
What the server actually does
The server is the orchestration boundary between clients and runtime infrastructure:
• It creates and deletes sandboxes.
• It returns connection info and endpoint metadata.
• It proxies requests into sandbox services.
• It manages pause/resume and expiration renewal.
• It enforces request authentication and validation.
• It abstracts over Docker and Kubernetes so clients see one API surface.
For an architecture diagram, this is the central “control plane” box.
4) Runtime-side components
The components/ directory contains executables and libraries that live closer to the sandbox runtime than the server does.
4.1 execd: execution daemon
components/execd/ is the runtime workhorse inside a sandbox. It provides APIs for:
• running commands
• handling sessions
• code interpretation / notebook-style execution
• file upload/download and file manipulation
• health checks and metrics
• SSE-based streaming of command/execution events
It also contains:
• Jupyter-related code paths
• runtime abstractions for bash sessions, process execution, interrupts, language handling, SQL, and context management
• a web router and controller layer
Architecturally, execd is the sandbox-local execution service. If you need to show “what happens after a sandbox exists,” this is the component that handles direct interaction with that sandbox.
4.2 ingress proxy
components/ingress/ is the inbound traffic gateway for sandboxes. It is responsible for:
• routing HTTP and WebSocket traffic to the right sandbox
• host-based routing and header handling
• health endpoints
• sandbox discovery / provider integration
• renewing sandbox intent when traffic is seen
This component sits on the network edge of sandbox access. It is the gateway through which external users reach sandbox-hosted services.
4.3 egress control
components/egress/ handles outbound network policy enforcement for sandboxes. It includes:
• DNS proxying and nameserver logic
• policy persistence and API handling
• nftables/iptables integration
• event broadcasting and webhook support
This is the outbound policy boundary: it controls what sandboxes are allowed to call on the network and how those rules are managed.
5) Kubernetes orchestration path
The kubernetes/ directory is a separate control plane for cluster-native sandbox execution.
Core pieces
• Custom resources:
- BatchSandbox
- Pool
• Controller binaries:
- cmd/controller
- cmd/task-executor
• Reconciler/controller logic in internal/controller
• Scheduler, allocator, eviction, and pool strategy logic
• Task executor runtime, server, and storage code
• Generated clientsets, informers, and listers for the CRDs
What this subsystem does
This is the scalable deployment path for sandbox workloads:
• The controller watches sandbox-related CRDs.
• It schedules and reconciles sandbox/pool state.
• It coordinates task placement and lifecycle.
• The task executor runs inside the cluster and performs the execution work for tasks.
• The executor has its own server, runtime abstractions, and local file store.
In diagram terms, the Kubernetes path is a second control plane with its own reconciliation loop:
• Custom resources enter the API server
• the controller reconciles desired state
• task executors perform work
• status is observed and fed back into the system
Important boundary
The Kubernetes subsystem is not just a deployment manifest directory. It includes:
• API types
• controller logic
• task execution runtime
• generated Kubernetes client code
• Helm charts and kustomize manifests
• e2e and runtime-specific tests
That makes it a full operator-style subsystem rather than just infra YAML.
6) Sandbox images and environment bundles
The sandboxes/ directory contains reference sandbox environments, such as the code-interpreter image and its startup scripts. These are the actual workload images that the server and Kubernetes runtime provision.
They are important because they sit between “platform control plane” and “user workload.” The platform creates them, but the actual execution environment lives here.
7) Examples, skills, and docs
These are not runtime subsystems, but they are important for understanding the intended architecture:
• examples/ shows how the platform is used for agents, browsers, desktop, code execution, and training.
• skills/ contains operational guidance, such as troubleshooting.
• docs/ contains architecture documentation and design notes.
• oseps/ contains proposal documents that explain planned or formalized subsystem changes.
8) Tests and validation
The repo has strong validation coverage across layers:
• unit tests inside each component
• server tests and k8s tests
• SDK tests in multiple languages
• cross-component E2E tests under tests/
This matters architecturally because the repo is designed around contract correctness between the server, runtime components, and SDKs. The tests are a sign that the boundaries are explicit and intentionally stable.
Main data flows
A. Sandbox creation and use
1. Application code calls a language SDK.
2. The SDK talks to the sandbox server lifecycle API.
3. The server creates a sandbox using Docker or Kubernetes.
4. The server returns sandbox metadata and connection details.
5. The SDK connects to the sandbox’s runtime endpoints, usually through execd or ingress.
6. Commands, files, and code execution requests are sent to the runtime service.
7. Streaming output and execution events are returned over HTTP/SSE.
B. Inbound user traffic to sandbox-hosted services
1. A client reaches the ingress proxy.
2. The ingress layer resolves the sandbox target.
3. The request is routed into the correct sandbox workload.
4. If configured, traffic can trigger renew-intent behavior to keep the sandbox alive.
C. Outbound sandbox traffic
1. The sandbox attempts external network access.
2. Egress control evaluates the request against policy.
3. DNS and packet-level controls enforce the decision.
4. Policy can be updated through the egress API.
D. Kubernetes execution path
1. A sandbox or pool resource is created as a CRD.
2. The controller reconciles desired state.
3. Scheduling/allocator logic selects placement.
4. A task executor runs the workload and reports status.
5. Client-facing APIs surface the resulting sandbox or task state.
How to draw the architecture
A clean diagram of this repo should show:
• Client applications and multi-language SDKs at the top
• OpenSandbox server as the central control plane
• A bifurcation from the server into Docker runtime and Kubernetes runtime paths
• Runtime-side services:
- execd for command/file/code execution
- ingress for inbound traffic routing
- egress for outbound policy enforcement
• Kubernetes controller and task-executor components as a separate cluster-native orchestration path
• Specs as the contract layer spanning SDKs, server, and components
• Sandbox images/environments as the workload layer beneath the control plane
The key architectural idea is that OpenSandbox is a platform with one API surface but multiple execution backends and runtime agents. The repository reflects that by separating contracts, control plane, runtime execution, and client SDKs into distinct subsystems.
Main entry points: opensandbox_server Server · tests E2E tests · services Services · specs Specs
Components 20
API contracts
Public interfaces that define platform boundaries
- Specs · Lifecycle, execd, and egress API definitions
specs
Client SDKs
Language SDKs and CLI entrypoints
- Sandbox SDKs · Python, JS, Kotlin, and C# clients for lifecycle and runtime APIs
sdks/sandbox - Code Interpreter SDKs · Higher-level code execution clients built on sandbox and execd contracts
sdks/code-interpreter - CLI · Command-line entrypoint for sandbox management
cli
Control plane
Central server and orchestration logic
- Server · Central API gateway, routing, and runtime abstraction
server/opensandbox_server - HTTP APIs · Lifecycle, proxy, pool, schema, and devops endpoints
server/opensandbox_server/api - Services · Backend service layer and runtime resolution
server/opensandbox_server/services - K8s service · Kubernetes-specific providers, templates, and helpers
server/opensandbox_server/services/k8s - Renew intent · Keeps sandboxes alive when active traffic is observed
server/opensandbox_server/integrations/renew_intent - Middleware · Authentication and request-id handling
server/opensandbox_server/middleware
Runtime components
Sandbox-local services and policy enforcement
Kubernetes path
Cluster-native orchestration and task execution
- Controller · Reconciles BatchSandbox and Pool custom resources
kubernetes/cmd/controller - Task executor · Runs sandbox tasks inside the cluster and reports status
kubernetes/cmd/task-executor - Reconcilers · Allocator, scheduler, eviction, and pool strategy logic
kubernetes/internal/controller - Task runtime · Local server, runtime adapters, and file storage for task execution
kubernetes/internal/task-executor - CRDs · BatchSandbox and Pool API types plus generated clients
kubernetes/apis/sandbox/v1alpha1
Workloads
Reference sandbox images and example usage
- Sandbox images · Reference sandbox environments and startup scripts
sandboxes/code-interpreter
Validation
End-to-end and subsystem tests
- E2E tests · Repository-wide end-to-end suites across SDKs, server, and Kubernetes
tests
Connections 30
- Specs → Sandbox SDKs: contracts · SDKs are generated from and aligned with public API specs
- Specs → Code Interpreter SDKs: contracts · Code interpreter clients follow the execd and sandbox contracts
- Specs → Server: contracts · Server implements the platform-facing APIs
- Specs → Execd: contracts · Runtime daemon exposes execd APIs
- Specs → Egress: contracts · Outbound policy APIs define egress behavior
- Sandbox SDKs → Server: lifecycle calls · Client SDKs create and manage sandboxes through the server
- Code Interpreter SDKs → Server: code calls · Code interpreter SDKs delegate execution setup through the platform
- CLI → Server: uses · CLI talks to the server for user-facing operations
- Server → HTTP APIs: serves · HTTP layer exposes lifecycle, proxy, pool, schema, and devops routes
- Server → Middleware: guards · Middleware handles auth and request identity
- Server → Services: orchestrates · Core service layer coordinates backends and runtime modes
- Services → Execd: proxies · Server routes execution and filesystem traffic to runtime endpoints
- Services → Ingress: routes through · Server surfaces sandbox endpoints that ingress later resolves
- Services → Egress: manages · Server exposes egress policy operations
- Services → K8s service: k8s mode · Kubernetes-specific service implementation
- Services → Renew intent: extends · Active traffic can trigger sandbox renewal
- Ingress → Renew intent: signals · Ingress traffic may keep sandboxes alive
- Ingress → Sandbox images: routes to · Edge proxy forwards traffic into running sandbox workloads
- Egress → Sandbox images: restricts · Outbound policy constrains sandbox workloads
- Controller → CRDs: watches · Controller reconciles sandbox custom resources
- Controller → Reconcilers: uses · Controller delegates scheduling and eviction decisions
- Reconcilers → Task executor: assigns tasks · Reconciliation results are executed by cluster workers
- Task executor → Task runtime: runs on · Task executor hosts local runtime and storage services
- Task runtime → Sandbox images: executes · Cluster tasks run workload images and reference environments
- E2E tests → Specs: validates · E2E coverage checks API contract behavior
- E2E tests → Server: validates · E2E suites exercise the control plane
- E2E tests → Execd: validates · Runtime tests cover execution daemon behavior
- E2E tests → Ingress: validates · Traffic routing is exercised end to end
- E2E tests → Egress: validates · Outbound policy behavior is tested end to end
- E2E tests → Controller: validates · Kubernetes orchestration is covered by integration suites