# mend3/browser-use architecture

> Architecture diagram of the GitHub repository mend3/browser-use, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mend3/browser-use
- Repository: https://github.com/mend3/browser-use

## Overview

##### Project type and purpose
- This is an open-source Python library that lets AI agents control real browsers. It exposes an Agent API that wraps an LLM (e.g. OpenAI, Claude) and drives Playwright under the hood, with DOM extraction, action controllers, telemetry, and optional UI integrations.

a. External LLM Service
- OpenAI, Claude, Gemini, Bedrock, etc.
- Communicates over HTTP/HTTPS (API calls).
b. Agent Layer (browser_use/agent)
- Agent core: orchestrates task, system + user prompts, planning loops.
- Message Manager: splits LLM replies into discrete action requests.
- Prompt templates & system_prompt.md.
c. Controller Layer (browser_use/controller)
- Registry of actions (click, fill, screenshot, custom functions).
- Service + views: decouple HTTP or UI bindings from core logic.
d. Browser Executor (browser_use/browser)
- Playwright wrapper (`Browser`, `Context`).
- Executes low-level commands, takes screenshots/GIFs.
e. DOM Processor (browser_use/dom)
- buildDomTree.js + Python service: extracts DOM state for the LLM.
- History tree processor: tracks mutations across steps.
f. Telemetry & Logging
- browser_use/telemetry: collects metrics, events.
- logging_config.py: central logging.
g. Integration & UI Examples (examples/ui, examples/integrations)
- Gradio, Streamlit front-ends.
- Slack/Discord bots.
h. Persistence & Artifacts
- GIF outputs, logs, custom-function outputs (files).

##### Interactions and data flow
1. The user creates an Agent(task, llm=ChatOpenAI…)
2. Agent composes a system+user prompt and calls the LLM API
3. LLM returns textual plan with actions
4. Message Manager parses the plan into discrete action messages
5. Controller looks up each action in its registry and invokes corresponding service
6. Browser Executor runs Playwright commands, updates the DOM state
7. DOM Processor extracts the updated DOM and converts it into a structured representation
8. Agent sends updated context back to LLM for next iteration
9. Telemetry logs each step; screenshots/GIFs optionally generated
10. Final result returned to user

##### Architectural patterns and principles
- Layered separation: Agent → Message Manager → Controller → Browser → DOM
- Pluggable registry: easy to add custom actions or integrate new endpoints
- Feedback loop: after each action, update state and re-invoke the LLM for planning
- Extensibility: custom-functions folder shows how to register user-provided logic

##### Technology highlights
- Python 3.11+, async/await for concurrency
- Playwright for browser automation
- buildDomTree.js for front-end DOM snapshot
- LangChain (optional) wrappers for LLMs
- Pytest for tests, pre-commit, GitHub Actions for CI/CD

##### Final checklist
- An inset showing how a developer would add a new action to the registry
- Keep the central LLM-agent-browser loop visually prominent

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "External Services"
        LLM["External LLM Service"]:::external
    end

    subgraph "Core Library"
        subgraph "Agent Layer"
            AgentCore["Agent Core"]:::agent
            MsgManager["Message Manager"]:::agent
            Prompts["Prompts & Templates"]:::agent
            AgentAPI["Agent API Views"]:::agent
            GIF["GIF Helper"]:::agent
        end

        subgraph "Controller Layer"
            RegService["Action Registry Service"]:::controller
            RegViews["Registry Views"]:::controller
            CtrlService["Controller Service"]:::controller
            CtrlViews["Controller Views"]:::controller
        end

        subgraph "Browser Executor"
            BrowserWrap["Browser & Context"]:::browser
            BrowserViews["Executor Views"]:::browser
        end

        subgraph "DOM Processor"
            BuildDOM["buildDomTree.js"]:::dom
            HistoryProc["History Tree Processor"]:::dom
            DOMService["DOM Service & Views"]:::dom
        end

        subgraph "Telemetry & Logging"
            TeleService["Telemetry Service"]:::telemetry
            TeleViews["Telemetry Views"]:::telemetry
            LogConfig["Logging Config"]:::telemetry
        end
    end

    subgraph "Integrations & UI"
        CMD["Command Line UI"]:::integration
        Gradio["Gradio Demo"]:::integration
        Streamlit["Streamlit Demo"]:::integration
        SlackAPI["Slack API"]:::integration
        SlackEx["Slack Example"]:::integration
        DiscordAPI["Discord API"]:::integration
        DiscordEx["Discord Example"]:::integration
        CustomFns["Custom-Functions"]:::integration
    end

    Artifacts["Persistence & Artifacts"]:::integration

    LLM -->|"prompt text"| AgentCore
    AgentCore -->|"uses templates"| Prompts
    AgentCore -->|"calls LLM API"| LLM
    AgentCore -->|"plan text"| MsgManager
    MsgManager -->|"action JSON"| RegService
    RegService -->|"invoke action"| CtrlService
    CtrlService -->|"Playwright commands"| BrowserWrap
    BrowserWrap -->|"DOM state"| BuildDOM
    BuildDOM -->|"DOM snapshot"| DOMService
    DOMService -->|"structured context"| AgentCore
    BrowserWrap -->|"screenshots/GIFs"| GIF
    BrowserWrap -->|"events"| TeleService
    DOMService -->|"events"| TeleService
    CtrlService -->|"events"| TeleService
    TeleService -->|"logs"| LogConfig
    AgentAPI -->|"API calls"| AgentCore
    CMD --> AgentAPI
    Gradio --> AgentAPI
    Streamlit --> AgentAPI
    SlackAPI --> AgentAPI
    SlackEx --> SlackAPI
    DiscordAPI --> AgentAPI
    DiscordEx --> DiscordAPI
    CustomFns --> RegService
    Artifacts -->|"stores files"| GIF

    subgraph "Developer Extensibility"
        CustomFns
        RegService
    end

    click AgentCore "https://github.com/mend3/browser-use/blob/main/browser_use/agent/service.py"
    click MsgManager "https://github.com/mend3/browser-use/blob/main/browser_use/agent/message_manager/service.py"
    click MsgManager "https://github.com/mend3/browser-use/blob/main/browser_use/agent/message_manager/utils.py"
    click MsgManager "https://github.com/mend3/browser-use/blob/main/browser_use/agent/message_manager/views.py"
    click Prompts "https://github.com/mend3/browser-use/blob/main/browser_use/agent/prompts.py"
    click Prompts "https://github.com/mend3/browser-use/blob/main/browser_use/agent/system_prompt.md"
    click AgentAPI "https://github.com/mend3/browser-use/blob/main/browser_use/agent/views.py"
    click GIF "https://github.com/mend3/browser-use/blob/main/browser_use/agent/gif.py"
    click RegService "https://github.com/mend3/browser-use/blob/main/browser_use/controller/registry/service.py"
    click RegViews "https://github.com/mend3/browser-use/blob/main/browser_use/controller/registry/views.py"
    click CtrlService "https://github.com/mend3/browser-use/blob/main/browser_use/controller/service.py"
    click CtrlViews "https://github.com/mend3/browser-use/blob/main/browser_use/controller/views.py"
    click BrowserWrap "https://github.com/mend3/browser-use/blob/main/browser_use/browser/browser.py"
    click BrowserWrap "https://github.com/mend3/browser-use/blob/main/browser_use/browser/context.py"
    click BrowserViews "https://github.com/mend3/browser-use/blob/main/browser_use/browser/views.py"
    click BuildDOM "https://github.com/mend3/browser-use/blob/main/browser_use/dom/buildDomTree.js"
    click HistoryProc "https://github.com/mend3/browser-use/blob/main/browser_use/dom/history_tree_processor/service.py"
    click HistoryProc "https://github.com/mend3/browser-use/blob/main/browser_use/dom/history_tree_processor/view.py"
    click DOMService "https://github.com/mend3/browser-use/blob/main/browser_use/dom/service.py"
    click DOMService "https://github.com/mend3/browser-use/blob/main/browser_use/dom/views.py"
    click TeleService "https://github.com/mend3/browser-use/blob/main/browser_use/telemetry/service.py"
    click TeleViews "https://github.com/mend3/browser-use/blob/main/browser_use/telemetry/views.py"
    click LogConfig "https://github.com/mend3/browser-use/blob/main/browser_use/logging_config.py"
    click CMD "https://github.com/mend3/browser-use/blob/main/examples/ui/command_line.py"
    click Gradio "https://github.com/mend3/browser-use/blob/main/examples/ui/gradio_demo.py"
    click Streamlit "https://github.com/mend3/browser-use/blob/main/examples/ui/streamlit_demo.py"
    click SlackAPI "https://github.com/mend3/browser-use/blob/main/examples/integrations/slack/slack_api.py"
    click SlackEx "https://github.com/mend3/browser-use/blob/main/examples/integrations/slack/slack_example.py"
    click DiscordAPI "https://github.com/mend3/browser-use/blob/main/examples/integrations/discord/discord_api.py"
    click DiscordEx "https://github.com/mend3/browser-use/blob/main/examples/integrations/discord/discord_example.py"
    click CustomFns "https://github.com/mend3/browser-use/tree/main/examples/custom-functions/"

    classDef external fill:#f9f,stroke:#333,stroke-width:1px
    classDef agent fill:#bbf,stroke:#333,stroke-width:1px
    classDef controller fill:#bfb,stroke:#333,stroke-width:1px
    classDef browser fill:#ffb,stroke:#333,stroke-width:1px
    classDef dom fill:#fbc,stroke:#333,stroke-width:1px
    classDef telemetry fill:#dbf,stroke:#333,stroke-width:1px
    classDef integration fill:#ffd,stroke:#333,stroke-width:1px
```
