# reworkd/bananalyzer architecture

> Architecture diagram of the GitHub repository reworkd/bananalyzer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/reworkd/bananalyzer
- Repository: https://github.com/reworkd/bananalyzer

## Overview

##### Project type and purpose
- Bananalyzer is an open-source Python CLI tool and library for evaluating AI “agents” on web-scraping tasks.
- It packages:
- A command-line interface (`__main__.py`)
- A core runner engine (`bananalyzer/runner/…`)
- A data layer for loading example definitions and MHTML snapshots (`bananalyzer/data/…`)
- A plugin interface (`AgentRunner`) that users implement to wrap their AI agents
- Optional FastAPI server components (documented but not present in the file tree)

##### Top-level file structure analysis
- `bananalyzer/` – core library
- `__main__.py`: CLI entrypoint
- `runner/`: orchestrates test generation & execution
- `data/`: loads example metadata & snapshots (local JSON, S3, schemas)
- `junit.py`, `hooks.py`, `schema.py`: reporting, event hooks, data models
- `static/` – store of MHTML site snapshots & example JSON definitions
- `tests/` – pytest test suite verifying each component
- Dev/config files: `pyproject.toml`, GitHub workflows, scripts

##### Key README insights
- CLI invokes pytest–based runner which:
a. Loads examples from JSON + MHTML
b. Dynamically builds a test suite
c. Spins up a Playwright `BrowserContext`
d. Calls `AgentRunner.run(context, example)` for each example
e. Collects an `AgentResult`, compares with expected outputs, emits JUnit XML
- Future FastAPI server serves example data via `/api/docs`.

##### Main components & interactions (
a. Components
- CLI (bananalyzer command)
- Core Runner Engine
– ExampleLoader (data layer)
– TestGenerator (pytest integration)
– Executor (loops over examples)
- Plugin Interface (AgentRunner implementations)
- Playwright BrowserContext (external service)
- Reporting (JUnit, console)
- Data Stores (static JSON, MHTML, optional S3 buckets)
- (Optional) FastAPI Server
b. Relationships & data flow
1. User → CLI
2. CLI → Runner Engine
3. Runner: load examples via ExampleLoader
4. Runner: create dynamic pytest tests (TestGenerator)
5. For each example:
– Instantiate Playwright context
– Call AgentRunner.run(context, example) → returns AgentResult
– Compare with expected via built-in eval functions
– Emit pass/fail → JUnit reporter
6. Runner aggregates results → console & XML output
7. (If enabled) FastAPI serves same example definitions via HTTP
c. Architectural patterns
- Plugin pattern for user agents (`AgentRunner`)
- Adapter/wrapper for Playwright API
- Factory for dynamic pytest testcases
- Layered separation: CLI → Core → Plugin → External
d. Technologies & frameworks
- Python 3.6+
- Playwright (async API)
- Pytest for test harness
- FastAPI/Uvicorn (server)
- JSON schemas for validation
- Optional S3 integration for remote snapshots

##### Detail level
- Sub-modules in Runner: `agent_runner.py`, `generator.py`, `evals.py`, `website_responder.py`
- Hook points (`hooks.py`) where custom logic can attach
- How tests map to pytest functions

## Diagram (Mermaid)

```mermaid
graph TB
    %% Subgraphs
    subgraph "CLI Layer"
        CLI["CLI Entry Point"]:::core
    end

    subgraph "Core Engine"
        Runner["Runner Orchestrator"]:::core
        TestGen["Dynamic Test Generator"]:::core
        Exec["Executor"]:::core
        Hooks["Event Hooks"]:::core
        Schema["Core Data Models"]:::core
    end

    subgraph "Data Layer"
        ELF["ExampleLoader\n(local JSON)"]:::core
        ELS3["ExampleLoader\n(S3)"]:::core
        ES["Example Schemas"]:::core
    end

    subgraph "Plugin Layer"
        AgentIF["Plugin Interface\n(AgentRunner)"]:::plugin
        NullAgent["NullAgentWrapper"]:::plugin
    end

    subgraph "External Services"
        Browser["Playwright BrowserContext"]:::external
        FastAPI["Optional FastAPI Server"]:::external
    end

    subgraph "Reporting & Extensions"
        Reporter["JUnit & Console Reporter"]:::core
    end

    subgraph "Data Stores"
        Static["Static Data Store\n(JSON & MHTML)"]:::datastore
        Tests["Test Suite"]:::datastore
    end

    %% Connections
    User(["User"]) -->|"invokes"| CLI
    CLI -->|"calls"| Runner
    Runner -->|"loadExamples"| ELF
    Runner -->|"loadExamples"| ELS3
    ELF -->|"returns ExampleDefinition"| Runner
    ELS3 -->|"returns ExampleDefinition"| Runner
    Runner -->|"validate"| ES
    Runner -->|"generateTests"| TestGen
    TestGen -->|"creates test cases"| Exec
    Exec -->|"instantiate"| Browser
    Exec -->|"invoke run(context,example)"| AgentIF
    AgentIF -->|"AgentResult"| Exec
    Exec -->|"evalResults"| Hooks
    Hooks -->|"hook modifications"| Exec
    Exec -->|"compare"| Reporter
    Runner -->|"aggregateResults"| Reporter
    Static -->|"feeds"| ELF
    Static -->|"feeds"| ELS3
    Tests -->|"verifies"| Runner
    FastAPI -->|"serves examples"| User

    %% Click Events
    click CLI "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/__main__.py"
    click Runner "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/runner/runner.py"
    click ELF "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/data/example_fetching.py"
    click ELS3 "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/data/example_s3.py"
    click ES "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/data/example_schemas.py"
    click ES "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/data/example_detail_schemas.py"
    click AgentIF "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/runner/agent_runner.py"
    click NullAgent "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/runner/null_agent_wrapper.py"
    click TestGen "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/runner/generator.py"
    click Exec "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/runner/evals.py"
    click Browser "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/runner/website_responder.py"
    click Reporter "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/junit.py"
    click Hooks "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/hooks.py"
    click Schema "https://github.com/reworkd/bananalyzer/blob/main/bananalyzer/schema.py"
    click Static "https://github.com/reworkd/bananalyzer/tree/main/static/"
    click Tests "https://github.com/reworkd/bananalyzer/tree/main/tests/"

    %% Styles
    classDef core fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef plugin fill:#90EE90,stroke:#000,stroke-width:1px
    classDef external fill:#FFA500,stroke:#000,stroke-width:1px
    classDef datastore fill:#D8BFD8,stroke:#000,stroke-width:1px
```
