# billy-enrizky/openbrowser-ai architecture

> Architecture diagram of the GitHub repository billy-enrizky/openbrowser-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/billy-enrizky/openbrowser-ai
- Repository: https://github.com/billy-enrizky/openbrowser-ai
- Video tour (about a minute): https://gitdiagram.com/billy-enrizky/openbrowser-ai/video

## Overview

OpenBrowser combines a Python browser-automation framework with a hosted task-running interface. Users submit browser tasks through the web UI or invoke the library, CLI daemon, or MCP server; agents use an LLM to plan or generate code, operate a CDP browser session, and return results. The hosted product adds task streaming, chat, projects, and VNC viewing. The repository also contains evaluation and model-training systems. Some frontend/backend and unsampled capability links are documented rather than visible in the excerpts, so those links are marked accordingly.

Main entry points: [`backend/app/services/agent_service.py`](https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/services/agent_service.py) (Hosted agent service), [`src/openbrowser/code_use/service.py`](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/code_use/service.py) (Code agent), [`src/openbrowser/agent/service.py`](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/agent/service.py) (Browser agent), [`backend/app/websocket/handler.py`](https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/websocket/handler.py) (Realtime task handler)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cloud["Hosted task platform"]
  node_web["Web dashboard<br/>[page.tsx]"]
  node_auth["Authentication UI<br/>[AuthProvider.tsx]"]
  node_stream["Task stream client<br/>[useTaskStream.ts]"]
  node_api["FastAPI application<br/>[main.py]"]
  node_task_api["Task endpoints<br/>[tasks.py]"]
  node_ws["Realtime task handler<br/>[handler.py]"]
  node_agent_service["Hosted agent service<br/>[agent_service.py]"]
  node_vnc["VNC session service<br/>[vnc_service.py]"]
  node_db[("Application database<br/>[models.py]")]
  node_schemas["API and task schemas<br/>[schemas.py]"]
end

subgraph group_agent["Agent runtime"]
  node_code_agent["Code agent<br/>[service.py]"]
  node_agent["Browser agent<br/>[service.py]"]
  node_llm["LLM providers"]
  node_tools["Browser actions<br/>[service.py]"]
  node_namespace["Persistent code namespace<br/>[namespace.py]"]
end

subgraph group_browser["Browser automation"]
  node_session["Browser session<br/>[session.py]"]
  node_dom["DOM extraction<br/>[service.py]"]
end

subgraph group_interfaces["Runtime interfaces"]
  node_daemon["CLI daemon<br/>[server.py]"]
  node_mcp["MCP server<br/>[server.py]"]
  node_extension["Browser extension<br/>[background.js]"]
end

subgraph group_research["Model research"]
  node_evaluation["Benchmark evaluation<br/>[eval_benchmark.py]"]
  node_training["Flow-matching training"]
end

node_user(("User"))

node_user -->|"submits tasks"| node_web
node_web -.->|"uses identity"| node_auth
node_web -.->|"streams progress"| node_stream
node_stream -.->|"connects to"| node_api
node_api -->|"routes requests"| node_task_api
node_api -->|"dispatches sockets"| node_ws
node_task_api -->|"uses task models"| node_schemas
node_task_api -->|"starts agents"| node_agent_service
node_ws -->|"runs tasks"| node_agent_service
node_ws -->|"persists messages"| node_db
node_agent_service -->|"sets up sessions"| node_vnc
node_agent_service -->|"creates agent"| node_code_agent
node_agent_service -->|"loads browser tools"| node_tools
node_user -->|"runs automation"| node_code_agent
node_code_agent -->|"executes code"| node_namespace
node_code_agent -->|"uses browser session"| node_session
node_agent -->|"requests decisions"| node_llm
node_agent -->|"reads browser state"| node_session
node_agent -->|"dispatches actions"| node_tools
node_tools -->|"operates browser"| node_session
node_session -->|"requests DOM data"| node_dom
node_user -->|"runs CLI tasks"| node_daemon
node_daemon -->|"executes code"| node_namespace
node_user -->|"invokes tools"| node_mcp
node_mcp -->|"executes code"| node_namespace
node_extension -->|"connects extension"| node_ws
node_evaluation -->|"benchmarks automation"| node_code_agent
node_training -.->|"trains browser models"| node_agent

click node_web "https://github.com/billy-enrizky/openbrowser-ai/blob/main/frontend/src/app/page.tsx"
click node_auth "https://github.com/billy-enrizky/openbrowser-ai/blob/main/frontend/src/components/auth/AuthProvider.tsx"
click node_stream "https://github.com/billy-enrizky/openbrowser-ai/blob/main/frontend/src/hooks/useTaskStream.ts"
click node_api "https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/main.py"
click node_task_api "https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/api/tasks.py"
click node_ws "https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/websocket/handler.py"
click node_agent_service "https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/services/agent_service.py"
click node_vnc "https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/services/vnc_service.py"
click node_db "https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/db/models.py"
click node_schemas "https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/models/schemas.py"
click node_code_agent "https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/code_use/service.py"
click node_agent "https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/agent/service.py"
click node_llm "https://github.com/billy-enrizky/openbrowser-ai/tree/main/src/openbrowser/llm"
click node_tools "https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/tools/service.py"
click node_namespace "https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/code_use/namespace.py"
click node_session "https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/browser/session.py"
click node_dom "https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/dom/service.py"
click node_daemon "https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/daemon/server.py"
click node_mcp "https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/mcp/server.py"
click node_extension "https://github.com/billy-enrizky/openbrowser-ai/blob/main/extension/background.js"
click node_evaluation "https://github.com/billy-enrizky/openbrowser-ai/blob/main/infra/eval/pipelines/eval_benchmark.py"
click node_training "https://github.com/billy-enrizky/openbrowser-ai/blob/main/infra/training/flow_matching/online_flow_llm_grpo_trainer.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web,node_auth,node_stream,node_api,node_task_api,node_ws,node_agent_service,node_vnc,node_db,node_schemas,node_user toneBlue
class node_code_agent,node_agent,node_llm,node_tools,node_namespace toneAmber
class node_session,node_dom toneMint
class node_daemon,node_mcp,node_extension toneRose
class node_evaluation,node_training toneIndigo
```

## Components

### Hosted task platform

- [Web dashboard](https://github.com/billy-enrizky/openbrowser-ai/blob/main/frontend/src/app/page.tsx): `frontend/src/app/page.tsx`
- [Authentication UI](https://github.com/billy-enrizky/openbrowser-ai/blob/main/frontend/src/components/auth/AuthProvider.tsx): `frontend/src/components/auth/AuthProvider.tsx`
- [Task stream client](https://github.com/billy-enrizky/openbrowser-ai/blob/main/frontend/src/hooks/useTaskStream.ts): `frontend/src/hooks/useTaskStream.ts`
- [FastAPI application](https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/main.py): `backend/app/main.py`
- [Task endpoints](https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/api/tasks.py): `backend/app/api/tasks.py`
- [Realtime task handler](https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/websocket/handler.py): `backend/app/websocket/handler.py`
- [Hosted agent service](https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/services/agent_service.py): `backend/app/services/agent_service.py`
- [VNC session service](https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/services/vnc_service.py): `backend/app/services/vnc_service.py`
- [Application database](https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/db/models.py): `backend/app/db/models.py`
- [API and task schemas](https://github.com/billy-enrizky/openbrowser-ai/blob/main/backend/app/models/schemas.py): `backend/app/models/schemas.py`

### Agent runtime

- [Code agent](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/code_use/service.py): `src/openbrowser/code_use/service.py`
- [Browser agent](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/agent/service.py): `src/openbrowser/agent/service.py`
- [LLM providers](https://github.com/billy-enrizky/openbrowser-ai/tree/main/src/openbrowser/llm): `src/openbrowser/llm`
- [Browser actions](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/tools/service.py): `src/openbrowser/tools/service.py`
- [Persistent code namespace](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/code_use/namespace.py): `src/openbrowser/code_use/namespace.py`

### Browser automation

- [Browser session](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/browser/session.py): `src/openbrowser/browser/session.py`
- [DOM extraction](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/dom/service.py): `src/openbrowser/dom/service.py`

### Runtime interfaces

- [CLI daemon](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/daemon/server.py): `src/openbrowser/daemon/server.py`
- [MCP server](https://github.com/billy-enrizky/openbrowser-ai/blob/main/src/openbrowser/mcp/server.py): `src/openbrowser/mcp/server.py`
- [Browser extension](https://github.com/billy-enrizky/openbrowser-ai/blob/main/extension/background.js): `extension/background.js`

### Model research

- [Benchmark evaluation](https://github.com/billy-enrizky/openbrowser-ai/blob/main/infra/eval/pipelines/eval_benchmark.py): `infra/eval/pipelines/eval_benchmark.py`
- [Flow-matching training](https://github.com/billy-enrizky/openbrowser-ai/blob/main/infra/training/flow_matching/online_flow_llm_grpo_trainer.py): `infra/training/flow_matching/online_flow_llm_grpo_trainer.py`

### Other components

- **User**

## Connections

- User → Web dashboard: submits tasks (evidence: `README.md`)
- Web dashboard → Authentication UI: uses identity
- Web dashboard → Task stream client: streams progress
- Task stream client → FastAPI application: connects to
- FastAPI application → Task endpoints: routes requests (evidence: `backend/app/main.py`)
- FastAPI application → Realtime task handler: dispatches sockets (evidence: `backend/app/main.py`)
- Task endpoints → API and task schemas: uses task models (evidence: `backend/app/api/tasks.py`)
- Task endpoints → Hosted agent service: starts agents (evidence: `backend/app/api/tasks.py`)
- Realtime task handler → Hosted agent service: runs tasks (evidence: `backend/app/websocket/handler.py`)
- Realtime task handler → Application database: persists messages (evidence: `backend/app/websocket/handler.py`)
- Hosted agent service → VNC session service: sets up sessions (evidence: `backend/app/services/agent_service.py`)
- Hosted agent service → Code agent: creates agent (evidence: `backend/app/services/agent_service.py`)
- Hosted agent service → Browser actions: loads browser tools (evidence: `backend/app/services/agent_service.py`)
- User → Code agent: runs automation (evidence: `README.md`)
- Code agent → Persistent code namespace: executes code (evidence: `src/openbrowser/code_use/service.py`)
- Code agent → Browser session: uses browser session (evidence: `src/openbrowser/code_use/service.py`)
- Browser agent → LLM providers: requests decisions (evidence: `src/openbrowser/agent/service.py`)
- Browser agent → Browser session: reads browser state (evidence: `src/openbrowser/agent/service.py`)
- Browser agent → Browser actions: dispatches actions (evidence: `src/openbrowser/agent/service.py`)
- Browser actions → Browser session: operates browser (evidence: `src/openbrowser/tools/service.py`)
- Browser session → DOM extraction: requests DOM data (evidence: `src/openbrowser/dom/service.py`)
- User → CLI daemon: runs CLI tasks (evidence: `README.md`)
- CLI daemon → Persistent code namespace: executes code (evidence: `src/openbrowser/daemon/server.py`)
- User → MCP server: invokes tools (evidence: `README.md`)
- MCP server → Persistent code namespace: executes code (evidence: `src/openbrowser/mcp/server.py`)
- Browser extension → Realtime task handler: connects extension
- Benchmark evaluation → Code agent: benchmarks automation (evidence: `infra/eval/pipelines/eval_benchmark.py`)
- Flow-matching training → Browser agent: trains browser models
