# samugit83/surfai architecture

> Architecture diagram of the GitHub repository samugit83/surfai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/samugit83/surfai
- Repository: https://github.com/samugit83/surfai

## Overview

##### Project Type and Purpose
- SurfAi is a full-stack web application that provides an AI-driven browser automation engine.
- It exposes a Flask-based web interface (front-end + back-end) that accepts natural-language instructions, translates them into Playwright commands, and drives a headless browser inside a Docker container.

##### Key Architectural Components
a. Front-end (UI)
- Served by Flask via `templates/index.html`, `static/styles.css`, `static/script.js`.
- Renders chat interface for user prompts and displays logs, screenshots, and progress.
b. Back-end (Flask + surf_ai engine)
- `app.py` + `config.py`: Flask app entrypoint, route definitions, environment config.
- `surf_ai/` modules:
- `engine.py`: Orchestrates the overall workflow (prompt → command generation → execution → self-correction).
- `prompt.py`: Manages LLM prompt templates and context assembly.
- `json_handler.py`: Parses LLM JSON responses into structured tasks.
- `command_executor.py`: Executes Playwright commands.
- `browser_manager.py`: Spawns/manages browser contexts and tabs.
- `element_highlighter.py`: Injects UI overlays for element numbering on the page.
- `screenshot_manager.py`: Captures and stores screenshots.
- `logging_handler.py`: Buffers and persists execution logs.
c. Data Layer
- `models/models.py`: Defines ORM-like objects to store tasks, execution state, and results in memory or Redis.
d. Persistence / Session Store
- Redis (as defined in `docker-compose.yml`) for session management and short-term buffering of context or chat memory.
e. Browser Automation
- Playwright launches a headless (or headed) browser in a sibling container.
- noVNC server (port 6901) exposes a VNC interface for visual debugging.
f. External Services
- OpenAI API for natural language understanding and command generation.

##### Relationships and Interactions
- User’s Browser → (HTTP) → Flask Server
- endpoints: `/` serves UI, `/api/chat` accepts prompts.
- Flask → surf_ai.engine → surf_ai.prompt → OpenAI API
- receives JSON response with action list.
- surf_ai.json_handler → surf_ai.command_executor → surf_ai.browser_manager → Playwright Browser
- executes actions, handles failures via self-correction.
- surf_ai.logging_handler + screenshot_manager → store logs/screenshots (could be local FS or Redis).
- surf_ai.models → persist task and result state in Redis.
- noVNC ↔ Playwright Browser for manual inspection (via web socket).

##### Architectural Patterns and Principles
- Modular “engine” components with single responsibility (prompt, parsing, execution, logging).
- Separation of concerns:
- Presentation layer (Flask templates/static)
- Orchestration layer (engine.py)
- Integration layer (OpenAI, Playwright).
- Retriable/self-correcting workflow: orchestrator re-invokes LLM on failures.
- Containerized microservices: web server, Redis, browser+noVNC.

##### Essential Elements to Capture
- Clear labels for each container/service/component.
- Callouts for critical design patterns (self-correcting loop, fallback strategies, interactive highlighting).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    subgraph "Client Layer"
        direction TB
        UI["Web UI\n(templates/index.html,\nstatic/styles.css,\nstatic/script.js)"]:::frontend
    end

    %% Server Layer
    subgraph "Web & API Server"
        direction TB
        Flask["Flask Server\n(app.py, config.py)"]:::backend
    end

    %% Automation Engine
    subgraph "Automation Engine" 
        direction TB
        Engine["Orchestrator\n(surf_ai/engine.py)"]:::engine
        Prompt["Prompt Manager\n(surf_ai/prompt.py)"]:::engine
        JSON["JSON Parser\n(surf_ai/json_handler.py)"]:::engine
        Executor["Command Executor\n(surf_ai/command_executor.py)"]:::engine
        BrowserMgr["Browser Manager\n(surf_ai/browser_manager.py)"]:::engine
        Highlighter["Element Highlighter\n(surf_ai/element_highlighter.py)"]:::engine
        Screenshot["Screenshot Manager\n(surf_ai/screenshot_manager.py)"]:::engine
        Logger["Logging Handler\n(surf_ai/logging_handler.py)"]:::engine
        Models["Models\n(models/models.py)"]:::datastore
    end

    %% External Services & Containers
    subgraph "External Services & Containers"
        direction TB
        OpenAI["OpenAI API"]:::external
        Redis[(Redis)]:::datastore
        subgraph "Browser Container"
            direction TB
            Playwright["Playwright Browser"]:::external
            noVNC["noVNC Server"]:::external
        end
    end

    %% Flows
    UI -->|"POST /api/chat"| Flask
    Flask -->|"invoke orchestrator"| Engine
    Engine -->|"build prompt"| Prompt
    Prompt -->|"call OpenAI"| OpenAI
    OpenAI -->|"JSON response"| JSON
    JSON -->|"tasks"| Executor
    Executor -->|"spawn context"| BrowserMgr
    BrowserMgr -->|"run steps"| Playwright
    Playwright -->|"render UI"| noVNC
    Executor -->|"highlight elements"| Highlighter
    Executor -->|"capture screenshots"| Screenshot
    Executor -->|"log events"| Logger
    Logger --> Models
    Executor --> Models
    Models --> Redis
    Flask --> Models

    %% Click Events
    click UI "https://github.com/samugit83/surfai/blob/master/templates/index.html"
    click UI "https://github.com/samugit83/surfai/blob/master/static/styles.css"
    click UI "https://github.com/samugit83/surfai/blob/master/static/script.js"
    click Flask "https://github.com/samugit83/surfai/blob/master/app.py"
    click Flask "https://github.com/samugit83/surfai/blob/master/config.py"
    click Engine "https://github.com/samugit83/surfai/blob/master/surf_ai/engine.py"
    click Prompt "https://github.com/samugit83/surfai/blob/master/surf_ai/prompt.py"
    click JSON "https://github.com/samugit83/surfai/blob/master/surf_ai/json_handler.py"
    click Executor "https://github.com/samugit83/surfai/blob/master/surf_ai/command_executor.py"
    click BrowserMgr "https://github.com/samugit83/surfai/blob/master/surf_ai/browser_manager.py"
    click Highlighter "https://github.com/samugit83/surfai/blob/master/surf_ai/element_highlighter.py"
    click Screenshot "https://github.com/samugit83/surfai/blob/master/surf_ai/screenshot_manager.py"
    click Logger "https://github.com/samugit83/surfai/blob/master/surf_ai/logging_handler.py"
    click Models "https://github.com/samugit83/surfai/blob/master/models/models.py"
    click Dockerfile "https://github.com/samugit83/surfai/tree/master/Dockerfile"
    click Compose "https://github.com/samugit83/surfai/blob/master/docker-compose.yml"
    click Entrypoint "https://github.com/samugit83/surfai/blob/master/entrypoint.sh"
    click Requirements "https://github.com/samugit83/surfai/blob/master/requirements.txt"

    %% Styles
    classDef frontend fill:#D0E8FF,stroke:#0366D6,color:#0366D6;
    classDef backend fill:#DFFFE0,stroke:#28A745,color:#22863A;
    classDef engine fill:#E8FFF4,stroke:#34D058,color:#22863A;
    classDef external fill:#FFEFD5,stroke:#D2691E,color:#A84300;
    classDef datastore fill:#F3E0FF,stroke:#6F42C1,color:#5A3E90
```
