# tinyfish-io/agentql architecture

> Architecture diagram of the GitHub repository tinyfish-io/agentql, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tinyfish-io/agentql
- Repository: https://github.com/tinyfish-io/agentql

## Overview

##### Main Components
- Core Query Engine: At the heart, show the module that interprets natural language queries, applies transformations, and maps queries to specific page elements.
- SDKs (Python and JavaScript): Represent these as client libraries that interact with the core engine via the REST API and wrap around Playwright integrations.
- REST API Endpoint: This acts as the interface for executing AgentQL queries. It bridges requests coming from SDKs, external integrations (like LangChain or Zapier), and any other client.
- Automation & Browser Driver Integration: Include Playwright (and possibly other automation tools) as the engine’s component that executes actions on real webpages.
- Debugger Browser Extension: As a tool for real-time query debugging and inspection, this component connects directly to the user’s browser.
- External Integrations: Show integration points for additional agent frameworks, automation tools, or even cloud services.

##### Relationships and Data Flows
- User Input flows from either a script (using the Python/JS SDK), a REST API call, or via the Debugger Extension.
- The SDKs package the natural language query and send it to the REST API.
- The REST API hands the query off to the core Query Engine.
- The Query Engine processes the query, interacts with the browser automation layer (via Playwright), and then returns results back down the chain.
- The Debugger Extension can also tap into this flow, enabling a live view of query execution and tweaking.
- External integrations (for example, with LangChain) interact through clearly defined API interfaces.

##### Architectural Patterns and Design Principles
- Separation of Concerns: Clearly delineate between the user interface (debugger extension, SDKs) and core backend logic (query engine, REST API, automation integration).
- Modular Design: Each component (SDKs, API, query engine, automation layer) is relatively independent, making it easy to extend or replace parts of the system.
- API-Driven Communication: The REST API serves as a contract between frontend clients (SDKs, debugger) and backend services.
- Reusability and Extensibility: The use of templates (in the .templates directory) suggests an emphasis on consistency, and the examples provided underline how common use cases may be rapidly prototyped.

##### Including Relevant Technologies
- Playwright: Emphasize its role in handling browser automation tasks.
- Python and JavaScript: Represent the client SDKs that use AgentQL queries.
- REST API Framework: Although not explicitly listed, include a block for the web server framework enabling the API endpoint.
- CI/CD/Testing Tools: Given the presence of configurations (.github workflows, .yamllint, Makefile), indicate a “DevOps/CI” component that automates testing, linting, and vulnerability scanning.

- Detailed sub-components inside the core system such as “AgentQL Query Engine,” “Natural Language Selector,” “Query Transformer,” and “Action Executor via Playwright.”
- Both Python and JavaScript SDK modules as separate components linked to the REST API.
- CI/CD & Testing Configurations might be abstracted into a “Deployment & Support Tools” block to illustrate how automated workflows (from .github/workflows and Makefile) integrate with development processes.

##### Detail and Clarity
- Avoid lumping components together unless they serve identical purposes; instead, break down the system into as many detailed nodes as possible.
- Every technology stack (SDKs, Playwright, REST API framework) is mentioned to capture the complete architectural overview.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Side
    subgraph "Client Side"
        UI("User Input / Client Application"):::client
        PSDK("Python SDK"):::client
        JSDK("JavaScript SDK"):::client
        DBG("Debugger Browser Extension"):::client
        EXT("External Integrations"):::external
    end

    %% Server Side
    subgraph "Server Side"
        API("REST API Endpoint"):::api
        ENG("AgentQL Query Engine"):::engine
        SEL("Natural Language Selector"):::engine
        TRANS("Query Transformer"):::engine
        EXEC("Action Executor via Playwright"):::engine
        AUTO("Browser Automation (Playwright)"):::automation
    end

    %% Deployment & Configuration
    subgraph "Deployment & Configuration"
        DEVOPS("CI/CD, Testing & Configuration"):::devops
        TEMPL("Templates & Project Configuration"):::devops
    end

    %% Connections from Client Side to Server Side
    UI -->|"sends_query"| PSDK
    UI -->|"sends_query"| JSDK
    UI -->|"debug_query"| DBG

    PSDK -->|"query_request"| API
    JSDK -->|"query_request"| API
    DBG -->|"debug_request"| API

    %% External Integration
    EXT -->|"integration_call"| API

    %% Server Side Processing
    API -->|"process_query"| ENG
    ENG -->|"select_language"| SEL
    ENG -->|"transform_query"| TRANS
    ENG -->|"execute_action"| EXEC
    EXEC -->|"invoke_browser_action"| AUTO

    %% DevOps Support (conceptual connection)
    DEVOPS ---|"supports"| ENG
    TEMPL ---|"configures"| ENG

    %% Click Events for Component Mapping
    click PSDK "https://github.com/tinyfish-io/agentql/tree/main/examples/python"
    click JSDK "https://github.com/tinyfish-io/agentql/tree/main/examples/js"
    click DEVOPS "https://github.com/tinyfish-io/agentql/blob/main/.github"
    click TEMPL "https://github.com/tinyfish-io/agentql/blob/main/.templates"
    
    %% Styles
    classDef client fill:#8ED1FC,stroke:#1F78C1,stroke-width:2px;
    classDef api fill:#F7A278,stroke:#D35400,stroke-width:2px;
    classDef engine fill:#FAD7A0,stroke:#E67E22,stroke-width:2px;
    classDef automation fill:#A9DFBF,stroke:#1ABC9C,stroke-width:2px;
    classDef external fill:#D2B4DE,stroke:#8E44AD,stroke-width:2px;
    classDef devops fill:#A3E4D7,stroke:#16A085,stroke-width:2px;
```
