How antonis19/autobrowse works
Main Components
• AutoBrowse Application (Entry Point): Represented by autobrowse.py; this initiates the overall flow.
• Planner Agent: The brain of the system that receives the task description from the user and coordinates the workflow. It may include functions such as ask_html_assistant() and ask_code_generator() to delegate subtasks.
• HTML Assistant Agent: Processes and simplifies the HTML from the browser. It chunks and filters HTML data to answer questions about the DOM.
• Code Generator Agent: Uses GPT-4 to generate puppeteer.js code for interacting with the browser (clicks, navigation, form filling). It interacts with the browser environment to send and adjust code as needed.
• Browser Console / Environment: Housed in the browser-console directory. It serves as a sandboxed browser that communicates via a websocket connection. This component accepts puppeteer code and returns rendered or processed HTML.
• External Services: Include integration with OpenAI’s APIs (via the configurations outlined in OAI_CONFIG_LIST) and potentially other dependencies (e.g., the autogen library mentioned in the README).
Relationships and Data Flow
• User provides a high-level task to the AutoBrowse application.
• The Planner Agent receives this task, breaks it down, and consults:
– The HTML Assistant to get context from the current page’s HTML (using preprocessing and chunking).
– The Code Generator to create precise puppeteer.js code to execute a specific browser action.
• The Code Generator sends the generated code to the Browser Environment (via the browser proxy or browser_console component) using a websocket connection.
• The Browser Environment executes the puppeteer.js commands (or other browser interactions) and returns the result (updated HTML or a confirmation).
• Data and error feedback loops (for example, if the code fails or if additional context is needed) are handled by the Planner, which coordinates additional calls to the agents.
• All these interactions are supported by OpenAI’s models as configured in agent_config.py and through the API key configuration file.
Architectural Patterns and Design Principles
• Separation of Concerns: Different responsibilities are clearly split among the Planner, HTML Assistant, and Code Generator, which is crucial for modularity.
• Agent-Based Orchestration: The design leverages specialized agents (each with a clear role) that coordinate through defined interfaces.
• Asynchronous Communication: The system uses a websocket connection between the browser console (sandboxed environment) and the agents for real-time communication.
• External Integration: Use of OpenAI APIs for natural language processing and code generation suggests a plug-and-play design for the AI components.