How operative-sh/web-eval-agent works
Project Type & Purpose
• This is a Python-based microservice (“MCP Server”) that exposes an autonomous web-evaluation agent.
• It integrates with Operative’s UVX CLI, Playwright for browser automation, and Cursor/Cline/Windsurf code editors.
• Its job is to launch a headless (or full) browser, inject an agent overlay, run end-to-end tests on a user’s web app, capture logs/network traffic, and return a structured report.
File Structure
• Entry point: webEvalAgent/mcp_server.py
• Core logic modules (webEvalAgent/src):
– api_utils.py: HTTP/request helpers
– tool_handlers.py: maps agent “tools” to actions
– prompts.py: agent-prompt management
– utils.py: shared helpers
• Browser orchestration: browser_manager.py, browser_utils.py
• Logging and traffic capture: log_server.py
• Environment setup: env_utils.py
• Agent overlay: src/agent_overlay.js (injected into the web page)
• Static UI for control center: templates/static/index.html
• Infrastructure/config: Dockerfile, .cursor/rules, pyproject.toml, requirements.txt, smithery.yaml, uv.lock
README for Insights
• Describes how UVX CLI calls the MCP Server using a JSON command in the editor.
• Highlights features: Navigate webapp, capture network, console errors, autonomous debugging loops.
• Shows example output report and use cases.
Main Components & Their Roles
a. Actors/External Services (Yellow):
– UVX CLI & Code Editor (Cursor/Cline/Windsurf)
– Operative API (for key validation/updates)
– Target Web App under test
– Playwright runtime (Node/npm)
b. MCP Server (Blue):
– mcp_server.py: HTTP or UVX entrypoint, orchestrates the flow
– api_utils, prompts, tool_handlers, utils: implement the agent’s decision logic
– env_utils: config/env loading
c. Browser Layer (Green):
– browser_manager: spawns and controls the browser via Playwright
– browser_utils: helper functions for page interactions
d. Instrumentation (Orange):
– log_server: intercepts console events & network requests
– agent_overlay.js: injected JS layer that exposes custom hooks to the agent
e. UI Layer (Purple):
– templates/static/index.html: Control Center dashboard for live logs
1. UVX CLI → MCP Server: “start web-eval” JSON command
2. MCP Server → browser_manager: “launch browser + inject agent_overlay.js”
3. browser_manager → Playwright: spawn, navigate to target URL
4. Playwright events → log_server: capture console logs & network traffic
5. log_server → MCP Server: push logs/events back to agent logic
6. MCP Server (tool_handlers) ↔ browser_manager: run clicks, type, navigation steps
7. MCP Server → Code Editor & Control Center UI: return final report
8. MCP Server → Operative API: optional calls to refresh packages or validate API key
Architectural Patterns & Design Principles
• Single Responsibility Modules: each .py handles a clear concern
• Event-Driven Flow: browser events drive agent decisions
• Plug-In Overlay: agent_overlay.js decouples in-page instrumentation from Python logic
• Microservice Packaging: Dockerfile for containerization, smithery.yaml/uv.lock for UVX deployment