# bjterry/boardgamebench architecture

> Architecture diagram of the GitHub repository bjterry/boardgamebench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bjterry/boardgamebench
- Repository: https://github.com/bjterry/boardgamebench

## Overview

##### Project Type and Purpose
- This is a backend‐centric evaluation framework (CLI tool + library) for running automated games between LLMs, recording results, and computing Elo ratings.
- Purpose: orchestrate matches, interact with LLM providers, persist match state and ratings in a database, and export experiment results.

##### High‐Level Components
a. CLI / Entry Point
- `bgbench/main.py`
- Supporting scripts: `init_postgres.py`, `generate_migration.py`
b. Configuration & Environment
- `.env`, `.env.example`, `config.py`, `bgbench/config_loader.py`
c. Data Layer (Persistence)
- ORM models & migrations: `bgbench/data/models.py`, Alembic configs (`alembic.ini`, `migrations/`)
- Database initialization/migration scripts: `bgbench/data/init_db.py`, `bgbench/data/migrate_db.py`
- Supported backends: SQLite (default) or PostgreSQL
d. Domain Logic
- Game abstractions & implementations: `bgbench/games/*.py` (e.g., chess, nim, poker)
- Match management: `bgbench/match/` (state manager, runner, view)
- Experiment orchestration: `bgbench/experiment/` (arena, scheduler, rating, export)
- LLM integration: `bgbench/llm/integration.py`, `bgbench/llm/player.py`
e. Rating Engine
- Bayesian Elo: `bgbench/experiment/rating.py`
f. Export & Reporting
- `bgbench/experiment/export.py`
g. Utilities
- Serialization: `bgbench/serialization.py`
- Logging config: `bgbench/logging_config.py`

##### External Dependencies & Services
- LLM Providers: OpenAI, Anthropic, OpenRouter (via litellm)
- Database: SQLite or PostgreSQL
- CLI / OS environment

##### Component Relationships and Data Flow
1. User invokes CLI (`bgbench/main.py`) with parameters (e.g., `--game nim`).
2. Config Loader reads environment and YAML experiment config.
3. Experiment Scheduler selects pairs of LLM players for matches.
4. Match Runner:
- Instantiates `Game` subclass + two `LLMPlayer` instances.
- Loops: Get move prompt → call LLM integration → validate moves → update `MatchState`.
- Stores intermediate and final states via Data Layer.
5. Rating Module consumes match results and updates Elo ratings in DB.
6. Exporter writes experiment summary to disk (CSV/JSON).
7. Logging at each stage for audit/troubleshooting.

##### Architectural Patterns & Principles
- Modular “plugin” pattern for games: each game is a subclass of `Game`, discovered via Python imports.
- Layered architecture: CLI → Orchestration (experiment) → Domain (games, matches) → Data Layer → External Services.
- Async support in scheduler/runner for concurrency.
- Separation of concerns: LLM integration decoupled from game logic.
- Use of Alembic for database migrations (infrastructure as code).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Layer
    subgraph "CLI & Entry Point"
        direction TB
        CLI["CLI / Entry Point\nbgbench/main.py"]:::internal
        InitDBScript["init_postgres.py"]:::internal
        GenMig["generate_migration.py"]:::internal
    end

    %% Configuration Layer
    subgraph "Configuration & Environment"
        direction TB
        EnvExample[".env.example"]:::internal
        ConfigPy["config.py"]:::internal
        ConfigLoader["Config Loader\nbgbench/config_loader.py"]:::internal
    end

    %% Orchestration Layer
    subgraph "Experiment Orchestration"
        direction TB
        Arena["Arena Orchestrator\nbgbench/experiment/arena.py"]:::internal
        Scheduler["Scheduler Strategies\nbgbench/experiment/scheduler.py"]:::internal
    end

    %% Domain Logic
    subgraph "Domain Logic"
        direction TB
        subgraph "Game Plugins"
            direction TB
            GameBase["Game Base\nbgbench/game.py"]:::internal
            Azul["azul_game.py"]:::internal
            Battleship["battleship_game.py"]:::internal
            CantStop["cant_stop_game.py"]:::internal
            Chess["chess_game.py"]:::internal
            GuessWho["guess_who_game.py"]:::internal
            LoveLetter["love_letter_game.py"]:::internal
            Nim["nim_game.py"]:::internal
            Poker["poker_game.py"]:::internal
            PokerTypes["poker_types.py"]:::internal
            Reversi["reversi_game.py"]:::internal
            Scrabble["scrabble_game.py"]:::internal
            War["war_game.py"]:::internal
        end
        subgraph "Match Management"
            direction TB
            MatchState["Match State\nbgbench/match/match_state.py"]:::internal
            Runner["Match Runner\nbgbench/match/runner.py"]:::internal
            StateMgr["State Manager\nbgbench/match/state_manager.py"]:::internal
            MatchView["Match View\nbgbench/match/view.py"]:::internal
        end
        subgraph "LLM Integration"
            direction TB
            LLMInt["Integration Layer\nbgbench/llm/integration.py"]:::internal
            LLMPlayer["LLM Player\nbgbench/llm/player.py"]:::internal
        end
    end

    %% Rating & Export
    subgraph "Rating & Export"
        direction TB
        Rating["Bayesian Elo Rating\nbgbench/experiment/rating.py"]:::internal
        Exporter["Exporter\nbgbench/experiment/export.py"]:::internal
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        Models["ORM Models\nbgbench/data/models.py"]:::internal
        InitDB["Init DB\nbgbench/data/init_db.py"]:::internal
        MigrateDB["Migrate DB\nbgbench/data/migrate_db.py"]:::internal
        AlembicCfg["alembic.ini"]:::internal
        MigrationsDir["migrations/"]:::internal
        DB["Database\nSQLite/PostgreSQL"]:::db
    end

    %% External
    subgraph "External Services"
        direction TB
        LLMProv["LLM Providers\n(OpenAI, Anthropic, OpenRouter)"]:::external
    end

    %% Utilities
    Serialization["Serialization\nbgbench/serialization.py"]:::internal
    LoggingCfg["Logging Config\nbgbench/logging_config.py"]:::internal

    %% Data Flow
    CLI -->|"reads env & config"| ConfigLoader
    CLI -->|"start experiment"| Arena
    Arena -->|"schedule matches"| Scheduler
    Scheduler -->|"match requests"| Runner
    Runner -->|"load game plugin"| GameBase
    GameBase --> GamePlugins
    Runner -->|"invoke player"| LLMPlayer
    LLMPlayer -->|"calls"| LLMInt
    LLMInt -.->|"async prompt"| LLMProv
    Runner -->|"update state"| StateMgr
    StateMgr -->|"persist state"| Models
    Models -->|"CRUD"| DB
    InitDB --> DB
    MigrateDB --> DB
    AlembicCfg --> MigrationsDir
    MigrationsDir --> DB
    Runner -->|"final results"| Rating
    Rating -->|"Elo update"| Models
    Rating --> DB
    Rating -->|"export summary"| Exporter
    Exporter -->|"write CSV/JSON"| CLI
    Serialization --> Runner
    LoggingCfg --> ALL

    %% Click Events
    click CLI "https://github.com/bjterry/boardgamebench/blob/main/bgbench/main.py"
    click InitDBScript "https://github.com/bjterry/boardgamebench/blob/main/init_postgres.py"
    click GenMig "https://github.com/bjterry/boardgamebench/blob/main/generate_migration.py"
    click EnvExample "https://github.com/bjterry/boardgamebench/blob/main/.env.example"
    click ConfigPy "https://github.com/bjterry/boardgamebench/blob/main/config.py"
    click ConfigLoader "https://github.com/bjterry/boardgamebench/blob/main/bgbench/config_loader.py"
    click Models "https://github.com/bjterry/boardgamebench/blob/main/bgbench/data/models.py"
    click InitDB "https://github.com/bjterry/boardgamebench/blob/main/bgbench/data/init_db.py"
    click MigrateDB "https://github.com/bjterry/boardgamebench/blob/main/bgbench/data/migrate_db.py"
    click AlembicCfg "https://github.com/bjterry/boardgamebench/blob/main/alembic.ini"
    click MigrationsDir "https://github.com/bjterry/boardgamebench/tree/main/migrations/"
    click GameBase "https://github.com/bjterry/boardgamebench/blob/main/bgbench/game.py"
    click Azul "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/azul_game.py"
    click Battleship "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/battleship_game.py"
    click CantStop "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/cant_stop_game.py"
    click Chess "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/chess_game.py"
    click GuessWho "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/guess_who_game.py"
    click LoveLetter "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/love_letter_game.py"
    click Nim "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/nim_game.py"
    click Poker "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/poker_game.py"
    click PokerTypes "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/poker_types.py"
    click Reversi "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/reversi_game.py"
    click Scrabble "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/scrabble_game.py"
    click War "https://github.com/bjterry/boardgamebench/blob/main/bgbench/games/war_game.py"
    click MatchState "https://github.com/bjterry/boardgamebench/blob/main/bgbench/match/match_state.py"
    click Runner "https://github.com/bjterry/boardgamebench/blob/main/bgbench/match/runner.py"
    click StateMgr "https://github.com/bjterry/boardgamebench/blob/main/bgbench/match/state_manager.py"
    click MatchView "https://github.com/bjterry/boardgamebench/blob/main/bgbench/match/view.py"
    click Arena "https://github.com/bjterry/boardgamebench/blob/main/bgbench/experiment/arena.py"
    click Scheduler "https://github.com/bjterry/boardgamebench/blob/main/bgbench/experiment/scheduler.py"
    click Rating "https://github.com/bjterry/boardgamebench/blob/main/bgbench/experiment/rating.py"
    click Exporter "https://github.com/bjterry/boardgamebench/blob/main/bgbench/experiment/export.py"
    click LLMInt "https://github.com/bjterry/boardgamebench/blob/main/bgbench/llm/integration.py"
    click LLMPlayer "https://github.com/bjterry/boardgamebench/blob/main/bgbench/llm/player.py"
    click Serialization "https://github.com/bjterry/boardgamebench/blob/main/bgbench/serialization.py"
    click LoggingCfg "https://github.com/bjterry/boardgamebench/blob/main/bgbench/logging_config.py"

    %% Styles
    classDef internal fill:#D0E6FF,stroke:#4A90E2,stroke-width:1px;
    classDef db fill:#DFFFE0,stroke:#49A942,stroke-width:1px;
    classDef external fill:#FFF4D0,stroke:#E29B00,stroke-width:1px,stroke-dasharray: 5 5;
    class ALL internal
```
