# red-sat/webact architecture

> Architecture diagram of the GitHub repository red-sat/webact, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/red-sat/webact
- Repository: https://github.com/red-sat/webact

## Overview

##### Main Components
- Agent Component: The core is the WebActAgent (implemented in webact/agent.py) which coordinates action prediction, browser navigation, state management, and interaction with various helper utilities.
- Utility and Helper Modules:
- Data Utilities (data_utils folder): Functions dedicated to prompt formatting and engineering.
- Demo Utilities (demo_utils folder): Components responsible for browser control (browser_helper.py), crawling (crawler_helper.py), prompt formatting (format_prompt.py), and model inference (inference_engine.py).
- External Tools/Services:
- Web browser automation via Playwright.
- Integration with external LLM services (e.g., Gemini 2.0, GPT-4o, LLaVA) as described in the README.
- Configuration/Testing Tools:
- quick_test.py for operational demonstrations.
- requirements.txt as dependency management.

##### Establish Relationships and Interactions
- The WebActAgent drives the overall workflow: it initiates and orchestrates operations by calling utility methods to generate and format prompts (data_utils) and execute automation tasks via browser helper and crawler (demo_utils).
- The Inference Engine in demo_utils/inference_engine.py interacts with the external AI models, effectively bridging between the agent’s prompt requests and receiving predictions.
- Data flows from the agent to the inference engine (for predictions), from the agent to the browser helper/crawler (for actual web operations), and back to the agent for decision-making and iteration.
- Configuration parameters, such as API keys and agent parameters (max_auto_op, grounding_strategy etc.), influence component behavior.

##### Architectural Patterns and Design Principles
- Modularity: Each module (agent, prompt formatting, browser control, crawling, inference) is separated into distinct directories to support a plug-and-play approach.
- Separation of Concerns: The project clearly demarcates the AI integration elements, the data transformation utilities, and the web navigation subprocess.
- Extensibility: Switching between different models (Gemini, GPT, LLaVA) and configuring tasks via parameters reveal a design ready for future expansion.
- Asynchronous operation: The provided usage example indicates asynchronous agent behavior in managing predictions and actions.

##### Relevant Technologies and Frameworks
- Python as the primary environment with dependencies managed through requirements.txt.
- Playwright for browser automation.
- Large Language Models via external API integrations (e.g., Gemini and GPT).
- Asynchronous programming (e.g., using async/await within agent operations).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Main Agent
    A["WebActAgent"]:::agent

    %% Data Utilities Subgraph
    subgraph "Data Utilities"
        D1["Format Prompt Utils"]:::utils
        D2["Prompts"]:::utils
    end

    %% Demo Utilities Subgraph
    subgraph "Demo Utilities"
        E1["Browser Helper"]:::utils
        E2["Crawler Helper"]:::utils
        E3["Demo Format Prompt"]:::utils
        E4["Inference Engine"]:::utils
    end

    %% External AI Services (using cylinder shape)
    X["External AI Services"]:::external

    %% Configuration & Testing Tools Subgraph
    subgraph "Configuration & Testing Tools"
        T1["Quick Test"]:::testing
        T2["Requirements"]:::testing
    end

    %% Connections
    A -->|"formats"| D1
    A -->|"uses"| D2
    A -->|"calls"| E3
    A -->|"drives"| E1
    A -->|"orchestrates"| E2
    A -->|"requests"| E4
    E4 -->|"queries"| X
    D1 -->|"provides"| E3
    D2 -->|"supports"| E3
    A -->|"configuredBy"| T1
    A -->|"dependsOn"| T2

    %% Click Events
    click A "https://github.com/red-sat/webact/blob/main/webact/agent.py"
    click D1 "https://github.com/red-sat/webact/blob/main/data_utils/format_prompt_utils.py"
    click D2 "https://github.com/red-sat/webact/blob/main/data_utils/prompts.py"
    click E1 "https://github.com/red-sat/webact/blob/main/demo_utils/browser_helper.py"
    click E2 "https://github.com/red-sat/webact/blob/main/demo_utils/crawler_helper.py"
    click E3 "https://github.com/red-sat/webact/blob/main/demo_utils/format_prompt.py"
    click E4 "https://github.com/red-sat/webact/blob/main/demo_utils/inference_engine.py"
    click T1 "https://github.com/red-sat/webact/blob/main/quick_test.py"
    click T2 "https://github.com/red-sat/webact/blob/main/requirements.txt"

    %% Styles
    classDef agent fill:#FFCCCC,stroke:#990000,stroke-width:2px,color:#000;
    classDef utils fill:#CCFFCC,stroke:#006600,stroke-width:2px,color:#000;
    classDef external fill:#CCE5FF,stroke:#004C99,stroke-width:2px,color:#000, shape: cylinder;
    classDef testing fill:#FFE6CC,stroke:#994C00,stroke-width:2px,color:#000;
```
