How lightpanda-io/browser works
1. Purpose and principal workflow. Lightpanda is a Zig-built headless browser for web automation and AI agents. An Automation client (for example Puppeteer over CDP) initiates the principal workflow by sending navigation and page-operation commands; Lightpanda creates an isolated browsing session, loads remote documents and resources, constructs and executes the page, then returns protocol results or serialized page output.
2. Components
Command, protocol, and agent interfaces
• Automation Client — null repository path: External CDP, WebDriver BiDi, or MCP consumer that requests browser contexts, pages, navigation, evaluation, and actions, as documented in the README.
• CLI User — null repository path: External operator invoking fetch, serve, agent, run, or mcp commands.
• Command Host — src/App.zig: Application-level command entry point, evidenced by the repository’s central application module and README command surface.
• CDP Server — src/server/cdp: Exposes Chrome DevTools Protocol and WebSocket automation access, documented as the server used by Puppeteer.
• WebDriver BiDi Server — src/server/bidi: Provides the optional WebDriver BiDi protocol selected through serve --protocol webdriver.
• MCP Server — src/mcp: Hosts MCP JSON-RPC over stdio or HTTP and, per documentation, maps clients to independently managed browsing sessions.
• Native Agent — src/agent/Agent.zig: Runs an interactive/task-driven agent in the browser process, issuing direct browser operations and supporting slash commands.
• LLM Provider — null repository path: External model endpoint used by agent mode; the README documents provider selection, OpenAI-compatible endpoints, gateways, and local-model alternatives.
• PandaScript Runner — src/browser/Runner.zig: Executes recorded deterministic JavaScript browser scripts for the run command.
Browser runtime and web state
• Browser Runtime — src/browser/Browser.zig: Top-level browser implementation that owns browser-wide runtime behavior.
• Browsing Session Manager — src/browser/Session.zig: Maintains session-level browsing context, including the isolation/sharing model described for MCP.
• Page Lifecycle — src/browser/Page.zig: Represents a page and its navigation-facing operations, including capture-oriented APIs referenced by the renderer.
• Frame and Document Runtime — src/browser/Frame.zig: Hosts frame-scoped document state and behavior needed for page loading and interaction.
• Resource Loader — src/browser/frame/resource_load.zig: Loads frame resources, a distinct stage from HTML parsing and script execution.
• Network Cache — src/network/cache: Provides network caching facilities indicated by the network subsystem layout.
• Adblock Filtering — src/network/adblock: Supplies request-filtering capability corresponding to the documented adblocker feature.
• Persistent Storage — src/storage/sqlite: SQLite-backed storage subsystem; the README documents browser cookies, while exact storage wiring was not sampled.
Document construction, URL processing, and JavaScript
• Zig HTML Parser Adapter — src/browser/parser/html5ever.zig: Browser-side adapter for the html5ever parser; the Rust prescanner explicitly identifies this as its synchronized counterpart.
• HTML5 Tree Builder — src/rust/html5ever/lib.rs: Exposes C-ABI document parsing, optional charset conversion, and html5ever tree construction to Zig.
• DOM Mutation Callback Sink — src/rust/html5ever/sink.rs: Implements html5ever’s tree-sink operations by invoking Zig-provided callbacks to create nodes, append text/nodes, reparent children, and attach declarative shadow roots.
• Script Preload Scanner — src/rust/html5ever/prescan.rs: Tokenizes buffered HTML without constructing a tree and reports eligible script/module URLs and the first base URL as prefetch hints.
• URL and IDNA Bridge — src/rust/html5ever/url.rs: Exposes URL parsing, joining, component access, and UTS#46 domain-to-ASCII conversion across the Zig/Rust boundary.
• V8 Isolate Integration — src/browser/js/Isolate.zig: Browser JavaScript-engine integration, consistent with the documented V8-based JavaScript support.
• Script Manager — src/browser/ScriptManager.zig: Manages page scripts as a separate lifecycle concern from parser and resource loading.
• Browser Actions — src/browser/actions.zig: Implements automation primitives such as interaction commands; README documents clicking and form input support.
Content extraction and rendering
• Semantic Content Tree — src/SemanticTree.zig: Represents semantic page content used for automation-oriented extraction/output.
• Page Dump Exporter — src/browser/dump.zig: Produces browser dump representations; README documents HTML and Markdown dumping.
• Screenshot Exporter — src/browser/screenshot.zig: Browser-side screenshot integration mirrored by the Rust rendering ABI.
• Text Rasterizer — src/rust/render/lib.rs: Lays out semantic text into styled blocks and rasterizes PNG output using bundled fonts, parley, skrifa, and tiny-skia; its source explicitly states it backs Page.captureScreenshot and --dump png.
3. Relationships
• Automation Client -> CDP Server: sends CDP/WebSocket automation requests; README’s Puppeteer example connects to the CDP endpoint.
• Automation Client -> WebDriver BiDi Server: sends WebDriver BiDi requests when that protocol is enabled; README documents the protocol option.
• Automation Client -> MCP Server: sends MCP JSON-RPC over stdio or HTTP; README specifies both transports and session headers.
• CLI User -> Command Host: invokes browser commands; README documents fetch, serve, agent, run, and mcp.
• Command Host -> CDP Server: starts the CDP serving mode; README documents lightpanda serve.
• Command Host -> WebDriver BiDi Server: starts BiDi as an alternative or co-enabled protocol; README documents repeated --protocol options.
• Command Host -> Native Agent: starts interactive or task-driven agent mode; README documents lightpanda agent.
• Command Host -> PandaScript Runner: dispatches recorded scripts for execution; README documents lightpanda run.
• MCP Server -> Browsing Session Manager: assigns, shares, lists, and closes browsing sessions; README explicitly defines this behavior.
• CDP Server -> Browser Runtime: exposes browser operations to automation clients; supported by the documented Puppeteer browser-context/page workflow.
• WebDriver BiDi Server -> Browser Runtime: exposes the same browser through its optional protocol; documented protocol role, with server wiring not sampled.
• Native Agent -> Browser Runtime: performs direct in-process browser tool calls; README explicitly states this.
• Native Agent -> LLM Provider: requests model assistance unless --no-llm is selected; README documents provider-backed and no-model modes.
• PandaScript Runner -> Browser Runtime: replays browser operations from saved JavaScript; README documents deterministic replay.
• Browser Runtime -> Browsing Session Manager: owns browsing-session behavior; supported by their runtime/session roles, though exact call sites were not sampled.
• Browsing Session Manager -> Page Lifecycle: manages pages within a browsing context; supported by documented context/new-page behavior.
• Page Lifecycle -> Frame and Document Runtime: hosts page document/frame behavior; supported by the page/frame subsystem organization.
• Frame and Document Runtime -> Resource Loader: requests document subresources; resource-loading module is frame-scoped by source organization.
• Resource Loader -> Network Cache: reads or writes cached network material; subsystem presence is established, but cache call sites were not sampled.
• Resource Loader -> Adblock Filtering: applies request filtering; README documents adblocking, but exact invocation was not sampled.
• Zig HTML Parser Adapter -> HTML5 Tree Builder: invokes the Rust html5ever integration; the prescanner source names the Zig adapter as its counterpart.
• HTML5 Tree Builder -> DOM Mutation Callback Sink: drives html5ever parsing through its supplied tree sink; Rust source constructs the sink and invokes document parsing.
• DOM Mutation Callback Sink -> Zig HTML Parser Adapter: invokes supplied node-creation and DOM-mutation callbacks; source shows the C callback ABI and callback calls.
• Script Preload Scanner -> Zig HTML Parser Adapter: reports script/base URL hints through a callback; source states the callback contract, while consuming call sites were not sampled.
• URL and IDNA Bridge -> Resource Loader: supplies URL parsing/normalization capabilities; source exposes this ABI, but direct loader calls were not sampled.
• Script Manager -> V8 Isolate Integration: executes managed page JavaScript through the browser’s V8 integration; component roles are clear, but direct call sites were not sampled.
• Page Lifecycle -> Browser Actions: performs interaction-oriented page operations; README documents click and form capabilities.
• Page Dump Exporter -> Semantic Content Tree: serializes semantic page content into output; roles are evident from the respective modules, but direct call sites were not sampled.
• Screenshot Exporter -> Text Rasterizer: requests text-only page rasterization; Rust source explicitly says it is mirrored by the screenshot exporter.
• Text Rasterizer -> Automation Client: returns screenshot content through page capture/protocol results; documented capture API role, with protocol serialization code not sampled.
4. Coverage limits. The sampled source is concentrated on Rust FFI parsing/rendering and optimization tools; Zig command dispatch, protocol handlers, network implementation, DOM ownership, storage use, and V8 call chains were not excerpted. Consequently, relationships marked as documented or role-based should not be read as verified function-level call paths, and provider-specific agent/authentication wiring cannot be established from the sample.
Main entry points: App.zig Command Host · Browser.zig Browser Runtime · resource_load.zig Resource Loader · Agent.zig Native Agent
Components 28
Interfaces
Command, protocol, and agent entry points
- Automation Client · External CDP, WebDriver BiDi, or MCP consumer
- CLI User · Operator invoking browser commands
- Command Host
src/App.zig - CDP Server
src/server/cdp - WebDriver BiDi Server · Optional WebDriver BiDi protocol
src/server/bidi - MCP Server · MCP JSON-RPC over stdio or HTTP
src/mcp - Native Agent
src/agent/Agent.zig - LLM Provider · External model endpoint
- PandaScript Runner
src/browser/Runner.zig
Browser Runtime
Browsing sessions, pages, frames, and network resources
- Browser Runtime
src/browser/Browser.zig - Browsing Session Manager
src/browser/Session.zig - Page Lifecycle
src/browser/Page.zig - Frame and Document Runtime
src/browser/Frame.zig - Resource Loader
src/browser/frame/resource_load.zig - Network Cache
src/network/cache - Adblock Filtering
src/network/adblock
Document and JavaScript
HTML parsing, URL processing, and script execution
- Zig HTML Parser Adapter
src/browser/parser/html5ever.zig - HTML5 Tree Builder
src/rust/html5ever/lib.rs - DOM Mutation Callback Sink
src/rust/html5ever/sink.rs - Script Preload Scanner
src/rust/html5ever/prescan.rs - URL and IDNA Bridge
src/rust/html5ever/url.rs - V8 Isolate Integration
src/browser/js/Isolate.zig - Script Manager
src/browser/ScriptManager.zig - Browser Actions
src/browser/actions.zig
Extraction and Rendering
Semantic extraction, dumps, and screenshots
- Semantic Content Tree
src/SemanticTree.zig - Page Dump Exporter
src/browser/dump.zig - Screenshot Exporter
src/browser/screenshot.zig - Text Rasterizer
src/rust/render/lib.rs
Connections 30
- Automation Client → CDP Server: sends CDP requests · CDP/WebSocket automation requests
- Automation Client → WebDriver BiDi Server: sends BiDi requests · Used when WebDriver BiDi is enabled
- Automation Client → MCP Server: sends MCP requests · MCP JSON-RPC over stdio or HTTP
- CLI User → Command Host: invokes commands · fetch, serve, agent, run, and mcp
- Command Host → CDP Server: starts serving
- Command Host → WebDriver BiDi Server: starts BiDi · Alternative or co-enabled protocol
- Command Host → Native Agent: starts agent
- Command Host → PandaScript Runner: dispatches scripts
- MCP Server → Browsing Session Manager: manages sessions · Assigns, shares, lists, and closes sessions
- CDP Server → Browser Runtime: exposes operations
- WebDriver BiDi Server → Browser Runtime: exposes operations · Optional protocol
- Native Agent → Browser Runtime: calls browser tools · Direct in-process browser operations
- Native Agent → LLM Provider: requests assistance · Optional unless no-LLM mode is selected
- PandaScript Runner → Browser Runtime: replays operations · Deterministic saved JavaScript replay
- Browser Runtime → Browsing Session Manager: owns sessions
- Browsing Session Manager → Page Lifecycle: manages pages
- Page Lifecycle → Frame and Document Runtime: hosts document
- Frame and Document Runtime → Resource Loader: requests resources · Frame document subresources
- Resource Loader → Network Cache: reads or writes
- Resource Loader → Adblock Filtering: applies filtering
- Zig HTML Parser Adapter → HTML5 Tree Builder: invokes parser · Rust html5ever integration
- HTML5 Tree Builder → DOM Mutation Callback Sink: drives tree sink
- DOM Mutation Callback Sink → Zig HTML Parser Adapter: invokes callbacks · Node creation and DOM mutation callbacks
- Script Preload Scanner → Zig HTML Parser Adapter: reports hints · Script and base URL callbacks
- URL and IDNA Bridge → Resource Loader: normalizes URLs · URL parsing and IDNA capabilities
- Script Manager → V8 Isolate Integration: executes scripts
- Page Lifecycle → Browser Actions: performs actions · Clicking and form input
- Page Dump Exporter → Semantic Content Tree: serializes content
- Screenshot Exporter → Text Rasterizer: requests rasterization · Text-only page rasterization
- Text Rasterizer → Automation Client: returns screenshots · Capture content through protocol results