# nkkko/ai-sandbox-benchmark architecture

> Architecture diagram of the GitHub repository nkkko/ai-sandbox-benchmark, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nkkko/ai-sandbox-benchmark
- Repository: https://github.com/nkkko/ai-sandbox-benchmark

## Overview

##### Main Components
- Benchmark Runner & CLI/TUI: The entry points are the “benchmark.py” (for the interactive terminal UI) and “comparator.py” (CLI mode). They coordinate test execution.
- Providers Module: Contains the “providers” directory with multiple provider-specific scripts (e.g., daytona.py, codesandbox.py, e2b.py, modal.py, local.py) and additional helper files (like utils.py) for dealing with different ecosystem integrations.
- Tests and Metrics: The “tests” directory holds a range of performance tests (e.g., test_calculate_primes.py, test_fft_performance.py, etc.). Supporting scripts such as “metrics.py” help capture and analyze performance data.
- Configuration & Utility Files: Files like “config.yml” and the “.env.example” outline configuration for environment variables and sandbox settings. Additionally, there are migration scripts and specification documents.

##### Relationships and Interactions
- The Benchmark Runner (benchmark.py) reads configuration details (via config.yml and environment variables), loads tests from the “tests” directory, and triggers parallel execution across the different providers.
- Each Provider module encapsulates the interface to the sandbox environments. For example, “daytona.py” manages Daytona-related operations, while “codesandbox.py” (and its associated codesandbox-service.js) handles CodeSandbox.
- The Tests execute independent benchmark routines, and the “metrics.py” file gathers performance metrics (workspace creation, code execution, cleanup, etc.) which can then be aggregated and displayed via the comparator.
- There is a clear data flow: Configuration → Benchmark Runner/TUI → Parallel test execution across Providers → Metrics Collection → Results display (CLI/TUI).

##### Architectural Patterns & Design Principles
- Modularity: The clear separation of the “providers”, “tests”, and the central benchmarking execution logic indicates a modular design that allows adding new providers or tests.
- Parallelism: Tests are run in parallel across all selected providers, emphasizing concurrency.
- Configuration Driven: The use of files like “config.yml” to determine what parameters are passed to different sandboxes demonstrates flexibility.
- Reusability and Extensibility: With separate files for rules (inside the “.cursor/rules” directory) and a well-documented providers folder, the architecture is designed to easily incorporate additional benchmarks or provider integrations.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Entry Points & UI
    subgraph "Entry Points & UI"
        BR["Benchmark Runner (benchmark.py)"]:::cli_ui
        CMP["Comparator (comparator.py)"]:::cli_ui
        UI["Terminal UI"]:::cli_ui
    end

    %% Configuration & Utility
    subgraph "Configuration & Utility"
        CFG["Configuration (config.yml)"]:::config
        ENV["Environment (.env.example)"]:::config
    end

    %% Tests & Metrics
    subgraph "Tests & Metrics"
        TS["Test Suite (tests/)"]:::test
        MET["Metrics Service (metrics.py)"]:::metric
    end

    %% Providers Module
    subgraph "Providers Module"
        PD["Provider: Daytona (daytona.py)"]:::provider
        PCS["Provider: CodeSandbox (codesandbox.py)"]:::provider
        PE2B["Provider: e2b (e2b.py)"]:::provider
        PMOD["Provider: Modal (modal.py)"]:::provider
        PLOC["Provider: Local (local.py)"]:::provider
        PUTILS["Providers Utils (utils.py)"]:::provider
        PCSS["Codesandbox Service (codesandbox-service.js)"]:::provider
    end

    %% Data Flow
    CFG -->|"reads"| BR
    ENV -->|"reads"| BR

    BR -->|"initiates tests"| TS
    BR -->|"invokes providers (parallel)"| PD
    BR -->|"invokes providers (parallel)"| PCS
    BR -->|"invokes providers (parallel)"| PE2B
    BR -->|"invokes providers (parallel)"| PMOD
    BR -->|"invokes providers (parallel)"| PLOC

    %% Provider internal dependency
    PCS -->|"utilizes"| PCSS
    PCS -->|"uses"| PUTILS

    TS -->|"executes and reports"| MET
    PD --> MET
    PCS --> MET
    PE2B --> MET
    PMOD --> MET
    PLOC --> MET

    MET -->|"displays results"| CMP
    CMP --> UI

    %% Styles
    classDef cli_ui fill:#ffcc99,stroke:#d35400,stroke-width:2px;
    classDef provider fill:#bbf0ff,stroke:#2980b9,stroke-width:2px;
    classDef test fill:#d2b4de,stroke:#8e44ad,stroke-width:2px;
    classDef metric fill:#fadbd8,stroke:#c0392b,stroke-width:2px;
    classDef config fill:#fcf3cf,stroke:#f1c40f,stroke-width:2px;

    %% Click Events
    click BR "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/benchmark.py"
    click CMP "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/comparator.py"
    click PD "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/providers/daytona.py"
    click PCS "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/providers/codesandbox.py"
    click PE2B "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/providers/e2b.py"
    click PMOD "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/providers/modal.py"
    click PLOC "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/providers/local.py"
    click PUTILS "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/providers/utils.py"
    click PCSS "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/providers/codesandbox-service.js"
    click TS "https://github.com/nkkko/ai-sandbox-benchmark/tree/main/tests/"
    click MET "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/metrics.py"
    click CFG "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/config.yml"
    click ENV "https://github.com/nkkko/ai-sandbox-benchmark/blob/main/.env.example"
```
