How takahirom/arbigent works
Project Type and Purpose
• The project is a full-fledged AI Agent testing framework – its purpose is to decompose complex UI testing into manageable scenarios with the help of AI.
• It supports multiple platforms (Android, iOS, Web, TV), and offers interchangeable AI providers (OpenAI, Gemini, Azure OpenAI), along with both UI and CLI testing interfaces.
File Structure
• The top-level project is organized into multiple modules (each a subproject) such as:
• arbigent-core and its associated packages (where most of the core business logic, agent orchestration, and task delegation reside).
• arbigent-ui (providing the graphical interface for test scenario creation and execution).
• arbigent-cli (offering a command-line interface to execute tests, integrated with GitHub Actions via CI configuration).
• arbigent-ai-openai (integrating the OpenAI provider with the system’s “ArbigentAi” interface).
• arbigent-core-model (common data types and shared models across the project).
• arbigent-core-web-report (handling web-based reporting, with custom webpack configurations).
• sample-test (example YAML files and tests to demonstrate end-to-end usage).
• Notice configuration files, Gradle build files, and GitHub workflows that indicate automated CI/CD pipelines and a modular build.
a. Main Components:
• Frontend/UI: Represented by arbigent-ui (desktop app) and the CLI component (arbigent-cli) – these serve as the two primary entry points for test execution.
• Core Engine: arbigent-core, which houses the main business logic including:
– ArbigentProject, ArbigentScenario, and ArbigentAgent for orchestrating test scenarios.
– Support classes such as ArbigentScenarioExecutor to manage workflow execution.
• AI Integration: Modules like arbigent-ai-openai provide concrete implementations of the ArbigentAi interface.
• Device Abstraction: The ArbigentDevice interface and its implementations (for Android, iOS, etc.) that abstract actions on physical/emulated devices.
• Persistence/Configuration: arbigent-core-model for data modeling and sample-test YAML project files.
• Reporting: arbigent-core-web-report generates reports, possibly in a web-based format.
• CI/CD and Build Pipelines: GitHub Actions workflows, which integrate testing (as seen in .github/workflows) into the development process.
b. Relationships and Interactions:
• The UI (or CLI) starts the testing process by sending an execution request to the core engine (ArbigentAgent).
• The core engine interacts in a loop with:
– ArbigentDevice: to fetch the current UI tree and perform actions on the actual device.
– ArbigentAi: to decide on the next action based on the provided goal, UI tree, and execution history.
• Communication flows are bidirectional; for example, the UI receives test results from the core engine.
c. Architectural Patterns and Design Principles:
• Modular Design: Separated functionality across multiple modules (CLI, UI, core, and AI integrations) promotes clear separation of concerns.
• Plugin/Interface Pattern: The AI provider (e.g., OpenAI, Gemini) is abstracted via the ArbigentAi interface, allowing extensibility.
• Orchestration Pattern: The framework decomposes complex tasks into sequential scenarios which depend on one another.
• Dependency Injection: Factories for AI and device implementations (as seen in code samples) indicate the use of dependency injection for flexibility.
d. Relevant Technologies and Frameworks:
• Gradle for building and managing dependencies.
• GitHub Actions for CI/CD, as indicated by workflow configurations.
• Kotlin for primary development in modules such as arbigent-ui, arbigent-cli, and arbigent-core.
• External APIs such as OpenAI and Gemini.
• Interceptors inspired by OkHttp’s pattern for customizing framework behaviors.
– Frontend Components: Arbigent UI and CLI.
– Core Engine: Detailing classes such as ArbigentAgent, ArbigentProject, ArbigentScenarioExecutor.
– Device Interface: Represent the ArbigentDevice abstraction and its implementations.
– AI Module: Represent ArbigentAi (with specific modules like arbigent-ai-openai).
– Reporting Module: (arbigent-core-web-report).
– Configuration/Persistence: YAML project files and shared core models (arbigent-core-model).
– CI/CD Pipelines: GitHub Actions integration.
• UI/CLI → Core Engine (test request and scenario execution).
• Core Engine → Device Interface (fetch UI tree, perform actions).
• Core Engine → AI Module (decide action and get results).
• Device Interface → Core Engine (return UI state and results).
• Core Engine → Reporting (send execution logs/results).