# energypantry/agent-browser-runtime architecture

> Architecture diagram of the GitHub repository energypantry/agent-browser-runtime, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/energypantry/agent-browser-runtime
- Repository: https://github.com/energypantry/agent-browser-runtime

## Overview

1. **What this repo is**
   - A compose-managed “real Chrome” runtime for AI agents.
   - The architecture is built around a visible, lease-based browser workspace instead of ephemeral headless sessions.

2. **Core system boundary: the Broker**
   - `broker/` is the control plane: Node/Fastify over HTTP + WebSocket.
   - It owns leases, jobs, artifacts, pacing/cooldowns, session state, and health reporting.
   - The broker is the main API surface other components talk to.

3. **Browser execution boundary: Chrome runtime**
   - `runtime/chrome/` packages Chromium/Chrome in Docker with Xvfb, x11vnc, noVNC, and startup scripts.
   - This is the actual browser execution environment, with a persistent profile mount so logins, cookies, localStorage, and extension state survive restarts.
   - Default host CDP port is `19223`, which is part of the browser-control boundary.

4. **Companion browser extension**
   - `extension/` is the in-browser agent bridge.
   - It owns real tabs and Tab Groups, can run debugger/CDP-style actions, capture screenshots/HTML, probe session state, and expose UI primitives such as move/click/type/scroll/wait.
   - It is the mechanism that lets the system operate through visible browser UI rather than synthetic backend shortcuts.

5. **Browser-to-control-plane flow**
   - The browser runtime loads the extension and exposes the live tab/session context.
   - The broker coordinates actions and state, while the extension performs the actual browser-side operations.
   - This split is important: the broker orchestrates; the extension executes inside Chrome.

6. **Optional TLS gateway path**
   - `tls-gateway/` is a separate Go service used as a local upstream proxy for Chromium’s startup networking path.
   - It is opt-in by default and surfaced in broker status when enabled.
   - Its role is transport consistency and proxy-chain control, not general application logic.

7. **Client/operator interface**
   - `cli/brs.js` is the operator client for the whole system.
   - It talks to the broker for status, one-shot fetches, continuous browsing leases, session probes, extractor runs, job inspection, artifact download, and cleanup.
   - This CLI is the easiest way to see the architecture from the outside: it exercises the broker API directly.

8. **Workload model: fetch, browse leases, and extractors**
   - The system supports two main browsing modes: one-shot `fetch` and persistent `browse-*` leases for multi-step workflows.
   - `extractors/` contains packaged site workflows that run against rendered pages and the broker/UI primitive layer.
   - Extractors return structured results while also allowing artifacts like HTML and screenshots to be captured for evidence and debugging.

9. **Artifacts and runtime state**
   - `artifacts/`, `data/`, and `runtime/profile/` are the important persistence boundaries.
   - Artifacts store outputs such as HTML, screenshots, probe results, and error evidence.
   - Runtime profile state preserves the browser identity/session between runs; this is central to the repo’s design.

10. **Deployment topology**
   - `docker-compose.yml` assembles the broker, browser runtime, and TLS gateway into one local stack.
   - `docker-compose.fingerprint.yml` is an overlay for swapping in a mounted fingerprinted Chromium binary.
   - `scripts/smoke-test.sh` validates the integrated stack end-to-end.

11. **Operational behavior and human handoff**
   - noVNC is a first-class human handoff channel for login, Captcha, MFA, and other account-safety checkpoints.
   - The repo emphasizes “real browser” pacing and UI interaction discipline rather than API shortcuts.
   - Platform cooldowns and humanized pacing are part of the runtime behavior, not just application logic.

12. **Documentation and usage contracts**
   - `docs/SPEC.md`, `docs/broker-api.md`, and `docs/docker-compose-architecture.md` define the intended contracts and deployment shape.
   - `skills/` contains Codex and OpenClaw skill packages describing how agents should use or deploy the runtime.
   - These docs are part of the architecture because they encode the intended control flow and boundaries between broker, browser, and operator.

Main entry points: [`broker/src/server.js`](https://github.com/energypantry/agent-browser-runtime/blob/main/broker/src/server.js) (Broker), [`runtime/chrome/entrypoint.sh`](https://github.com/energypantry/agent-browser-runtime/blob/main/runtime/chrome/entrypoint.sh) (Chrome runtime), [`extension/background.js`](https://github.com/energypantry/agent-browser-runtime/blob/main/extension/background.js) (Extension), [`docker-compose.yml`](https://github.com/energypantry/agent-browser-runtime/blob/main/docker-compose.yml) (Compose stack)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control["Control plane"]
  node_cli["CLI<br/>[brs.js]"]
  node_broker["Broker<br/>control plane<br/>[server.js]"]
  node_store[("State store<br/>broker state<br/>[store.js]")]
  node_ext_rpc["Extension RPC<br/>broker bridge<br/>[extension-rpc.js]"]
  node_health(("Healthcheck<br/>broker probe<br/>[healthcheck.js]"))
  node_extractors["Extractors<br/>workflow pack<br/>[example.extract.js]"]
end

subgraph group_browser["Browser runtime"]
  node_chrome_runtime["Chrome runtime<br/>browser container<br/>[entrypoint.sh]"]
  node_launch["Launch script<br/>browser startup<br/>[launch-browser.sh]"]
  node_supervisor["Supervisor<br/>process manager<br/>[supervisord.conf]"]
  node_extension["Extension<br/>browser agent bridge<br/>[background.js]"]
  node_runtime_cfg["Runtime config<br/>extension config<br/>[runtime-config.js]"]
  node_stealth["Stealth script<br/>extension content<br/>[stealth-content.js]"]
  node_tls_gateway["TLS gateway<br/>upstream proxy<br/>[tls-gateway.go]"]
end

subgraph group_ops["Operations"]
  node_artifacts[("Artifacts<br/>output store<br/>[.gitkeep]")]
  node_data[("Data<br/>persistent state<br/>[.gitkeep]")]
  node_profile[("Browser profile<br/>persistent profile")]
  node_compose["Compose stack<br/>deployment<br/>[docker-compose.yml]"]
  node_fingerprint["Fingerprint overlay<br/>compose overlay"]
  node_smoke["Smoke test<br/>validation<br/>[smoke-test.sh]"]
end

subgraph group_knowledge["Docs and skills"]
  node_docs["Docs<br/>[SPEC.md]"]
  node_skills["Skills<br/>usage contracts<br/>[SKILL.md]"]
end

node_cli -->|"API"| node_broker
node_broker -->|"persists"| node_store
node_broker -->|"commands"| node_ext_rpc
node_broker -->|"reports"| node_health
node_broker -->|"writes"| node_artifacts
node_broker -->|"reads/writes"| node_data
node_broker -->|"runs"| node_extractors
node_broker -.->|"uses"| node_tls_gateway
node_chrome_runtime -->|"starts"| node_launch
node_chrome_runtime -->|"supervises"| node_supervisor
node_chrome_runtime -->|"loads"| node_extension
node_chrome_runtime -->|"mounts"| node_profile
node_chrome_runtime -.->|"routes"| node_tls_gateway
node_extension -->|"config"| node_runtime_cfg
node_extension -->|"injects"| node_stealth
node_extension -->|"captures"| node_artifacts
node_extension -->|"syncs"| node_store
node_extractors -->|"uses"| node_extension
node_extractors -->|"invoked by"| node_broker
node_compose -->|"assembles"| node_broker
node_compose -->|"assembles"| node_chrome_runtime
node_compose -.->|"optionally includes"| node_tls_gateway
node_fingerprint -.->|"overlays"| node_compose
node_smoke -->|"validates"| node_compose
node_docs -.->|"specifies"| node_broker
node_docs -.->|"describes"| node_compose
node_skills -.->|"guides"| node_broker
node_skills -.->|"guides"| node_chrome_runtime

click node_cli "https://github.com/energypantry/agent-browser-runtime/blob/main/cli/brs.js"
click node_broker "https://github.com/energypantry/agent-browser-runtime/blob/main/broker/src/server.js"
click node_store "https://github.com/energypantry/agent-browser-runtime/blob/main/broker/src/store.js"
click node_ext_rpc "https://github.com/energypantry/agent-browser-runtime/blob/main/broker/src/extension-rpc.js"
click node_health "https://github.com/energypantry/agent-browser-runtime/blob/main/broker/src/healthcheck.js"
click node_chrome_runtime "https://github.com/energypantry/agent-browser-runtime/blob/main/runtime/chrome/entrypoint.sh"
click node_launch "https://github.com/energypantry/agent-browser-runtime/blob/main/runtime/chrome/launch-browser.sh"
click node_supervisor "https://github.com/energypantry/agent-browser-runtime/blob/main/runtime/chrome/supervisord.conf"
click node_extension "https://github.com/energypantry/agent-browser-runtime/blob/main/extension/background.js"
click node_runtime_cfg "https://github.com/energypantry/agent-browser-runtime/blob/main/extension/runtime-config.js"
click node_stealth "https://github.com/energypantry/agent-browser-runtime/blob/main/extension/stealth-content.js"
click node_tls_gateway "https://github.com/energypantry/agent-browser-runtime/blob/main/tls-gateway/tls-gateway.go"
click node_extractors "https://github.com/energypantry/agent-browser-runtime/blob/main/extractors/example.extract.js"
click node_artifacts "https://github.com/energypantry/agent-browser-runtime/blob/main/artifacts/.gitkeep"
click node_data "https://github.com/energypantry/agent-browser-runtime/blob/main/data/.gitkeep"
click node_compose "https://github.com/energypantry/agent-browser-runtime/blob/main/docker-compose.yml"
click node_fingerprint "https://github.com/energypantry/agent-browser-runtime/blob/main/docker-compose.fingerprint.yml"
click node_smoke "https://github.com/energypantry/agent-browser-runtime/blob/main/scripts/smoke-test.sh"
click node_docs "https://github.com/energypantry/agent-browser-runtime/blob/main/docs/SPEC.md"
click node_skills "https://github.com/energypantry/agent-browser-runtime/blob/main/skills/codex/agent-browser-runtime/SKILL.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_broker,node_store,node_ext_rpc,node_health,node_extractors toneBlue
class node_chrome_runtime,node_launch,node_supervisor,node_extension,node_runtime_cfg,node_stealth,node_tls_gateway toneAmber
class node_artifacts,node_data,node_profile,node_compose,node_fingerprint,node_smoke toneMint
class node_docs,node_skills toneRose
```

## Components

### Control plane

Broker, operator client, and workflow contracts

- [CLI](https://github.com/energypantry/agent-browser-runtime/blob/main/cli/brs.js): operator client · Operator entrypoint for broker status, fetch, browse, probes, extractors, and cleanup · `cli/brs.js`
- [Broker](https://github.com/energypantry/agent-browser-runtime/blob/main/broker/src/server.js): control plane · Fastify HTTP/WebSocket service that owns leases, jobs, artifacts, pacing, sessions, and health · `broker/src/server.js`
- [State store](https://github.com/energypantry/agent-browser-runtime/blob/main/broker/src/store.js): broker state · Broker persistence for leases, jobs, runtime identity, and session data · `broker/src/store.js`
- [Extension RPC](https://github.com/energypantry/agent-browser-runtime/blob/main/broker/src/extension-rpc.js): broker bridge · Broker-side adapter for browser extension messaging and UI primitive execution · `broker/src/extension-rpc.js`
- [Healthcheck](https://github.com/energypantry/agent-browser-runtime/blob/main/broker/src/healthcheck.js): broker probe · Broker health and readiness reporting · `broker/src/healthcheck.js`
- [Extractors](https://github.com/energypantry/agent-browser-runtime/blob/main/extractors/example.extract.js): workflow pack · Packaged site workflows that run against rendered pages and capture structured results · `extractors/example.extract.js`

### Browser runtime

Chrome container, extension bridge, and browser-side execution

- [Chrome runtime](https://github.com/energypantry/agent-browser-runtime/blob/main/runtime/chrome/entrypoint.sh): browser container · Dockerized Chromium environment with persistent profile and visible browser session · `runtime/chrome/entrypoint.sh`
- [Launch script](https://github.com/energypantry/agent-browser-runtime/blob/main/runtime/chrome/launch-browser.sh): browser startup · Browser startup orchestration and runtime wiring · `runtime/chrome/launch-browser.sh`
- [Supervisor](https://github.com/energypantry/agent-browser-runtime/blob/main/runtime/chrome/supervisord.conf): process manager · Keeps browser-side services running inside the runtime container · `runtime/chrome/supervisord.conf`
- [Extension](https://github.com/energypantry/agent-browser-runtime/blob/main/extension/background.js): browser agent bridge · In-browser agent layer for tabs, tab groups, screenshots, HTML, and UI primitives · `extension/background.js`
- [Runtime config](https://github.com/energypantry/agent-browser-runtime/blob/main/extension/runtime-config.js): extension config · Extension-side runtime configuration and environment wiring · `extension/runtime-config.js`
- [Stealth script](https://github.com/energypantry/agent-browser-runtime/blob/main/extension/stealth-content.js): extension content · Content script for browser-side stealth and page adaptation · `extension/stealth-content.js`
- [TLS gateway](https://github.com/energypantry/agent-browser-runtime/blob/main/tls-gateway/tls-gateway.go): upstream proxy · Optional Go proxy for Chromium startup networking and proxy-chain control · `tls-gateway/tls-gateway.go`

### Operations

Persistence, deployment, and validation

- [Artifacts](https://github.com/energypantry/agent-browser-runtime/blob/main/artifacts/.gitkeep): output store · Captured HTML, screenshots, probes, and error evidence · `artifacts/.gitkeep`
- [Data](https://github.com/energypantry/agent-browser-runtime/blob/main/data/.gitkeep): persistent state · Shared runtime data and persisted operational state · `data/.gitkeep`
- **Browser profile**: persistent profile · Mounted browser identity, cookies, localStorage, and extension state
- [Compose stack](https://github.com/energypantry/agent-browser-runtime/blob/main/docker-compose.yml): deployment · Local stack assembly for broker, browser runtime, and optional TLS gateway · `docker-compose.yml`
- [Fingerprint overlay](https://github.com/energypantry/agent-browser-runtime/blob/main/docker-compose.fingerprint.yml): compose overlay · Alternate compose overlay for fingerprinted Chromium binary · `docker-compose.fingerprint.yml`
- [Smoke test](https://github.com/energypantry/agent-browser-runtime/blob/main/scripts/smoke-test.sh): validation · End-to-end stack validation script · `scripts/smoke-test.sh`

### Docs and skills

Architecture docs and usage contracts

- [Docs](https://github.com/energypantry/agent-browser-runtime/blob/main/docs/SPEC.md): architecture docs · Specification, broker API, and compose architecture contracts · `docs/SPEC.md`
- [Skills](https://github.com/energypantry/agent-browser-runtime/blob/main/skills/codex/agent-browser-runtime/SKILL.md): usage contracts · Codex and OpenClaw operator/deploy guidance · `skills/codex/agent-browser-runtime/SKILL.md`

## Connections

- CLI → Broker: API · Operator client exercises the broker control surface
- Broker → State store: persists · Broker owns leases, jobs, sessions, and pacing state
- Broker → Extension RPC: commands · Broker delegates browser-side actions through extension RPC
- Broker → Healthcheck: reports · Broker exposes readiness and health status
- Broker → Artifacts: writes · Broker stores captured outputs and evidence
- Broker → Data: reads/writes · Broker uses shared operational state and runtime data
- Broker → Extractors: runs · Broker coordinates one-shot and lease-based workflows
- Broker → TLS gateway: uses · Broker surfaces optional gateway status when enabled
- Chrome runtime → Launch script: starts · Container entrypoint launches the browser stack
- Chrome runtime → Supervisor: supervises · Runtime container keeps browser-side services alive
- Chrome runtime → Extension: loads · Chrome runtime loads the agent bridge extension
- Chrome runtime → Browser profile: mounts · Persistent browser profile survives restarts
- Chrome runtime → TLS gateway: routes · Optional startup networking can flow through the gateway
- Extension → Runtime config: config · Extension behavior is parameterized by runtime config
- Extension → Stealth script: injects · Content script adapts page behavior inside browser tabs
- Extension → Artifacts: captures · Extension produces screenshots, HTML, and probe evidence
- Extension → State store: syncs · Extension state is coordinated with broker session state
- Extractors → Extension: uses · Packaged workflows rely on browser-side UI primitives
- Extractors → Broker: invoked by · Extractors run under broker-managed workflows
- Compose stack → Broker: assembles · Compose brings up the control plane
- Compose stack → Chrome runtime: assembles · Compose brings up the browser runtime
- Compose stack → TLS gateway: optionally includes · TLS gateway is part of the local stack when enabled
- Fingerprint overlay → Compose stack: overlays · Fingerprint stack modifies the base compose topology
- Smoke test → Compose stack: validates · Smoke test checks the integrated stack end to end
- Docs → Broker: specifies · Docs define broker API and contracts
- Docs → Compose stack: describes · Docs explain the deployment shape
- Skills → Broker: guides · Skills describe how operators use the runtime via broker
- Skills → Chrome runtime: guides · Skills describe deployment and browser runtime usage
