How tinyfish-io/agentql works
Main Components
• Core Query Engine: At the heart, show the module that interprets natural language queries, applies transformations, and maps queries to specific page elements.
• SDKs (Python and JavaScript): Represent these as client libraries that interact with the core engine via the REST API and wrap around Playwright integrations.
• REST API Endpoint: This acts as the interface for executing AgentQL queries. It bridges requests coming from SDKs, external integrations (like LangChain or Zapier), and any other client.
• Automation & Browser Driver Integration: Include Playwright (and possibly other automation tools) as the engine’s component that executes actions on real webpages.
• Debugger Browser Extension: As a tool for real-time query debugging and inspection, this component connects directly to the user’s browser.
• External Integrations: Show integration points for additional agent frameworks, automation tools, or even cloud services.
Relationships and Data Flows
• User Input flows from either a script (using the Python/JS SDK), a REST API call, or via the Debugger Extension.
• The SDKs package the natural language query and send it to the REST API.
• The REST API hands the query off to the core Query Engine.
• The Query Engine processes the query, interacts with the browser automation layer (via Playwright), and then returns results back down the chain.
• The Debugger Extension can also tap into this flow, enabling a live view of query execution and tweaking.
• External integrations (for example, with LangChain) interact through clearly defined API interfaces.
Architectural Patterns and Design Principles
• Separation of Concerns: Clearly delineate between the user interface (debugger extension, SDKs) and core backend logic (query engine, REST API, automation integration).
• Modular Design: Each component (SDKs, API, query engine, automation layer) is relatively independent, making it easy to extend or replace parts of the system.
• API-Driven Communication: The REST API serves as a contract between frontend clients (SDKs, debugger) and backend services.
• Reusability and Extensibility: The use of templates (in the .templates directory) suggests an emphasis on consistency, and the examples provided underline how common use cases may be rapidly prototyped.
Including Relevant Technologies
• Playwright: Emphasize its role in handling browser automation tasks.
• Python and JavaScript: Represent the client SDKs that use AgentQL queries.
• REST API Framework: Although not explicitly listed, include a block for the web server framework enabling the API endpoint.
• CI/CD/Testing Tools: Given the presence of configurations (.github workflows, .yamllint, Makefile), indicate a “DevOps/CI” component that automates testing, linting, and vulnerability scanning.
• Detailed sub-components inside the core system such as “AgentQL Query Engine,” “Natural Language Selector,” “Query Transformer,” and “Action Executor via Playwright.”
• Both Python and JavaScript SDK modules as separate components linked to the REST API.
• CI/CD & Testing Configurations might be abstracted into a “Deployment & Support Tools” block to illustrate how automated workflows (from .github/workflows and Makefile) integrate with development processes.
Detail and Clarity
• Avoid lumping components together unless they serve identical purposes; instead, break down the system into as many detailed nodes as possible.
• Every technology stack (SDKs, Playwright, REST API framework) is mentioned to capture the complete architectural overview.