How billy-enrizky/openbrowser-ai works
OpenBrowser combines a Python browser-automation framework with a hosted task-running interface. Users submit browser tasks through the web UI or invoke the library, CLI daemon, or MCP server; agents use an LLM to plan or generate code, operate a CDP browser session, and return results. The hosted product adds task streaming, chat, projects, and VNC viewing. The repository also contains evaluation and model-training systems. Some frontend/backend and unsampled capability links are documented rather than visible in the excerpts, so those links are marked accordingly.
Main entry points: agent_service.py Hosted agent service · code_use/service.py Code agent · agent/service.py Browser agent · handler.py Realtime task handler
Components 23
Hosted task platform
- Web dashboard
frontend/src/app/page.tsx - Authentication UI
frontend/src/components/auth/AuthProvider.tsx - Task stream client
frontend/src/hooks/useTaskStream.ts - FastAPI application
backend/app/main.py - Task endpoints
backend/app/api/tasks.py - Realtime task handler
backend/app/websocket/handler.py - Hosted agent service
backend/app/services/agent_service.py - VNC session service
backend/app/services/vnc_service.py - Application database
backend/app/db/models.py - API and task schemas
backend/app/models/schemas.py
Agent runtime
- Code agent
src/openbrowser/code_use/service.py - Browser agent
src/openbrowser/agent/service.py - LLM providers
src/openbrowser/llm - Browser actions
src/openbrowser/tools/service.py - Persistent code namespace
src/openbrowser/code_use/namespace.py
Browser automation
- Browser session
src/openbrowser/browser/session.py - DOM extraction
src/openbrowser/dom/service.py
Runtime interfaces
- CLI daemon
src/openbrowser/daemon/server.py - MCP server
src/openbrowser/mcp/server.py - Browser extension
extension/background.js
Model research
- Benchmark evaluation
infra/eval/pipelines/eval_benchmark.py - Flow-matching training
infra/training/flow_matching/online_flow_llm_grpo_trainer.py
Other components
- User
Connections 28
- User → Web dashboard: submits tasks
README.md - Web dashboard → Authentication UI: uses identity
- Web dashboard → Task stream client: streams progress
- Task stream client → FastAPI application: connects to
- FastAPI application → Task endpoints: routes requests
backend/app/main.py - FastAPI application → Realtime task handler: dispatches sockets
backend/app/main.py - Task endpoints → API and task schemas: uses task models
backend/app/api/tasks.py - Task endpoints → Hosted agent service: starts agents
backend/app/api/tasks.py - Realtime task handler → Hosted agent service: runs tasks
backend/app/websocket/handler.py - Realtime task handler → Application database: persists messages
backend/app/websocket/handler.py - Hosted agent service → VNC session service: sets up sessions
backend/app/services/agent_service.py - Hosted agent service → Code agent: creates agent
backend/app/services/agent_service.py - Hosted agent service → Browser actions: loads browser tools
backend/app/services/agent_service.py - User → Code agent: runs automation
README.md - Code agent → Persistent code namespace: executes code
src/openbrowser/code_use/service.py - Code agent → Browser session: uses browser session
src/openbrowser/code_use/service.py - Browser agent → LLM providers: requests decisions
src/openbrowser/agent/service.py - Browser agent → Browser session: reads browser state
src/openbrowser/agent/service.py - Browser agent → Browser actions: dispatches actions
src/openbrowser/agent/service.py - Browser actions → Browser session: operates browser
src/openbrowser/tools/service.py - Browser session → DOM extraction: requests DOM data
src/openbrowser/dom/service.py - User → CLI daemon: runs CLI tasks
README.md - CLI daemon → Persistent code namespace: executes code
src/openbrowser/daemon/server.py - User → MCP server: invokes tools
README.md - MCP server → Persistent code namespace: executes code
src/openbrowser/mcp/server.py - Browser extension → Realtime task handler: connects extension
- Benchmark evaluation → Code agent: benchmarks automation
infra/eval/pipelines/eval_benchmark.py - Flow-matching training → Browser agent: trains browser models