How servicenow/browsergym works
Main Components
• Core Functionality Module: Represent the “browsergym/core” directory. This is the foundation responsible for key functions such as environment setup, task management, action handling, and providing common utilities (for example, the abstraction in the AbstractBrowserTask class that benchmarks inherit).
• Benchmarks/Extensions: Include all separate benchmark modules such as:
– AssistantBench (browsergym/assistantbench)
– MiniWoB (browsergym/miniwob)
– VisualWebArena (browsergym/visualwebarena)
– WebArena (browsergym/webarena)
– Experiments (browsergym/experiments)
These modules extend the core to create specialized environments.
• Demo Agent: Highlight the “demo_agent” folder that contains a runnable agent script (run_demo.py), the demo configuration (environment.yml), and instructions for launching an agent linked to the underlying environments.
• Testing and CI/CD: Incorporate the “tests” directory, which holds test suites for core modules and each benchmark, and note the presence of GitHub workflows (in .github/workflows) and other CI-related files for continuous integration.
• Documentation & Development Tools: Indicate the related documentation (docs folder, README files in submodules) and supporting configuration files (Makefile, pyproject.toml, pre-commit config, etc.).
Relationships and Interactions
• Experiment & Demo Layers: Above the benchmarks, show modules for “Experiments” and “Demo Agent” that provide sample agents, loops, and usage examples which orchestrate the interaction with the benchmarking environments.
• Testing & CI/CD: Include a “Tests” block that interacts with all modules. Additionally, show a CI/CD pipeline block (GitHub Actions) that validates code on pushes and pull requests.
• External Dependencies: Optionally, note external libraries (like gymnasium, Playwright, etc.) that support rendering, browser control, or agent functionality.
Architectural Patterns and Design Principles
• Modularity/Extensibility: Emphasize the decoupled design as benchmarks are kept in separate directories but inherit from a common core, allowing for easy extension and integration.
• Use of Open Standards: Notice how the project leverages the Gymnasium API for environment creation and the plugin style for benchmarks, which should be represented by clearly defined interfaces between modules.
• Layered Architecture: Clearly separate lower-level core functionalities from higher-level user-facing demos/experiments and tests.
Technologies, Frameworks, and Tools
• Python ecosystem (pyproject.toml and Makefile-based installation),
• GitHub Workflows (for CI/CD),
• Documentation generation (ReadTheDocs configuration),
• Dependency management (requirements.txt for each component),
• Web automation (Playwright for Chromium installation).