How samugit83/surfai works
Project Type and Purpose
• SurfAi is a full-stack web application that provides an AI-driven browser automation engine.
• It exposes a Flask-based web interface (front-end + back-end) that accepts natural-language instructions, translates them into Playwright commands, and drives a headless browser inside a Docker container.
Key Architectural Components
a. Front-end (UI)
• Served by Flask via templates/index.html, static/styles.css, static/script.js.
• Renders chat interface for user prompts and displays logs, screenshots, and progress.
b. Back-end (Flask + surf_ai engine)
• app.py + config.py: Flask app entrypoint, route definitions, environment config.
• surf_ai/ modules:
• engine.py: Orchestrates the overall workflow (prompt → command generation → execution → self-correction).
• prompt.py: Manages LLM prompt templates and context assembly.
• json_handler.py: Parses LLM JSON responses into structured tasks.
• command_executor.py: Executes Playwright commands.
• browser_manager.py: Spawns/manages browser contexts and tabs.
• element_highlighter.py: Injects UI overlays for element numbering on the page.
• screenshot_manager.py: Captures and stores screenshots.
• logging_handler.py: Buffers and persists execution logs.
c. Data Layer
• models/models.py: Defines ORM-like objects to store tasks, execution state, and results in memory or Redis.
d. Persistence / Session Store
• Redis (as defined in docker-compose.yml) for session management and short-term buffering of context or chat memory.
e. Browser Automation
• Playwright launches a headless (or headed) browser in a sibling container.
• noVNC server (port 6901) exposes a VNC interface for visual debugging.
f. External Services
• OpenAI API for natural language understanding and command generation.
Relationships and Interactions
• User’s Browser → (HTTP) → Flask Server
• endpoints: / serves UI, /api/chat accepts prompts.
• Flask → surf_ai.engine → surf_ai.prompt → OpenAI API
• receives JSON response with action list.
• surf_ai.json_handler → surf_ai.command_executor → surf_ai.browser_manager → Playwright Browser
• executes actions, handles failures via self-correction.
• surf_ai.logging_handler + screenshot_manager → store logs/screenshots (could be local FS or Redis).
• surf_ai.models → persist task and result state in Redis.
• noVNC ↔ Playwright Browser for manual inspection (via web socket).
Architectural Patterns and Principles
• Modular “engine” components with single responsibility (prompt, parsing, execution, logging).
• Separation of concerns:
• Presentation layer (Flask templates/static)
• Orchestration layer (engine.py)
• Integration layer (OpenAI, Playwright).
• Retriable/self-correcting workflow: orchestrator re-invokes LLM on failures.
• Containerized microservices: web server, Redis, browser+noVNC.
Essential Elements to Capture
• Clear labels for each container/service/component.
• Callouts for critical design patterns (self-correcting loop, fallback strategies, interactive highlighting).