# browser-use/workflow-use architecture

> Architecture diagram of the GitHub repository browser-use/workflow-use, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/browser-use/workflow-use
- Repository: https://github.com/browser-use/workflow-use

## Overview

##### Project type and purpose
- This is a hybrid tool consisting of:
- A browser extension (built with TypeScript/React/Tailwind, bundled via Vite) that records user interactions and generates “workflow” JSON files.
- A Python‐based CLI & library (workflow_use) that loads those JSON workflows, executes them deterministically (with automatic LLM fallback/self-healing), and interacts with a browser automation engine (Browser Use/Playwright).
- Purpose: Record once, replay reliably, with structured variables and self-healing.

##### File structure
- Top-level:
- extension/ – all code for the browser extension: entrypoints (background.ts, content.ts, sidepanel UI), shared lib & types, assets, components.json, config (vite.config.ts, wxt.config.ts).
- workflows/ – Python code & CLI: cli.py, pyproject.toml, modules under workflow_use (builder, controller, recorder, schema, workflow), example workflows, tests, .env for OpenAI key.
- static/, LICENSE, README.md, workflows/.env.example.
- Patterns:
- Extension uses a message-bus type system for content⇄background⇄sidepanel communication.
- Python package structured by domain: builder (construct workflows), recorder (capture execution), controller (runtime orchestration), workflow (core execution logic), schema (validation).

##### README for additional insights
- The README details the two phases:
a. Build the extension → bundle for Chrome/Edge etc.
b. Setup Python env → run CLI commands (`run-as-tool`, `run-workflow`, `create-workflow`).
- Dependencies: Node/NPM, Playwright (Chromium), Python3, OpenAI API.

##### Important: be very detailed
- Message-bus types and entrypoints in the extension.
- Self-healing loop: Controller → LLM → update workflow → retry step.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Browser Extension Layer
    subgraph "Browser Extension" 
        direction TB
        UI["Sidepanel UI"]:::frontend
        BG["Background Script"]:::frontend
        CS["Content Script"]:::frontend
        MB["Message Bus & Shared Types"]:::frontend
    end

    %% Storage Artifact
    WFJSON["Workflow JSON Artifact"]:::storage

    %% Python CLI & Library
    subgraph "Python CLI & Library" 
        direction TB
        CLI["CLI Entrypoint"]:::backend
        subgraph "Builder Module"
            direction TB
            B_prompts["prompts.py"]:::backend
            B_service["service.py"]:::backend
        end
        subgraph "Recorder Module"
            direction TB
            R_core["recorder.py"]:::backend
            R_service["service.py"]:::backend
            R_views["views.py"]:::backend
        end
        subgraph "Controller Module"
            direction TB
            C_service["service.py"]:::backend
            C_utils["utils.py"]:::backend
            C_views["views.py"]:::backend
        end
        subgraph "Workflow Core Module"
            direction TB
            W_service["service.py"]:::backend
            W_prompts["prompts.py"]:::backend
            W_views["views.py"]:::backend
        end
        subgraph "Schema/Validation"
            direction TB
            S_views["views.py"]:::backend
        end
    end

    %% External Services
    Playwright["Browser Use / Playwright"]:::external
    LLM["OpenAI LLM API"]:::external

    %% Relationships
    User["User"]:::external -->|"interacts"| UI
    UI -->|"postMessage"| BG
    BG -->|"dispatch"| CS
    BG -->|"uses"| MB
    CS -->|"captures DOM events"| MB
    MB -->|"emits JSON"| WFJSON

    CLI -->|"reads JSON"| WFJSON
    CLI -->|"instantiates"| B_prompts
    CLI -->|"calls builder.service"| B_service

    B_service -->|"builds workflow object"| W_service
    W_service -->|"uses schema.views"| S_views

    CLI -->|"calls recorder.recorder"| R_core
    CLI -->|"uses recorder.service"| R_service
    R_service -->|"renders views"| R_views

    R_core -->|"executes steps via"| Playwright
    Playwright -->|"actions events"| CS

    CLI -->|"orchestrates via"| C_service
    C_service -->|"onFailure calls"| LLM
    LLM -->|"returns fix"| C_utils
    C_utils -->|"updates workflow"| W_service
    C_service -->|"renders"| C_views

    W_service -->|"runs workflow"| Playwright

    WFJSON -->|"updated JSON"| CLI

    %% Click Events
    click BG "https://github.com/browser-use/workflow-use/blob/main/extension/src/entrypoints/background.ts"
    click CS "https://github.com/browser-use/workflow-use/blob/main/extension/src/entrypoints/content.ts"
    click UI "https://github.com/browser-use/workflow-use/blob/main/extension/src/entrypoints/sidepanel/index.tsx"
    click MB "https://github.com/browser-use/workflow-use/blob/main/extension/src/lib/message-bus-types.ts"
    click MB "https://github.com/browser-use/workflow-use/blob/main/extension/src/lib/workflow-types.ts"
    click MB "https://github.com/browser-use/workflow-use/blob/main/extension/src/lib/utils.ts"
    click MB "https://github.com/browser-use/workflow-use/blob/main/extension/src/lib/types.ts"
    click WFJSON "https://github.com/browser-use/workflow-use/blob/main/workflows/examples/example.workflow.json"
    click CLI "https://github.com/browser-use/workflow-use/blob/main/workflows/cli.py"
    click B_prompts "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/builder/prompts.py"
    click B_service "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/builder/service.py"
    click R_core "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/recorder/recorder.py"
    click R_service "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/recorder/service.py"
    click R_views "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/recorder/views.py"
    click C_service "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/controller/service.py"
    click C_utils "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/controller/utils.py"
    click C_views "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/controller/views.py"
    click W_service "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/workflow/service.py"
    click W_prompts "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/workflow/prompts.py"
    click W_views "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/workflow/views.py"
    click S_views "https://github.com/browser-use/workflow-use/blob/main/workflows/workflow_use/schema/views.py"

    %% Styles
    classDef frontend fill:#D0E8FF,stroke:#0366d6,color:#0366d6
    classDef backend fill:#D8F8D8,stroke:#28a745,color:#28a745
    classDef external fill:#FFE8A0,stroke:#FF8800,color:#A65E00
    classDef storage fill:#F0F0F0,stroke:#666666,color:#333333
```
