# aidinhamedi/optimizer-benchmark architecture

> Architecture diagram of the GitHub repository aidinhamedi/optimizer-benchmark, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aidinhamedi/optimizer-benchmark
- Repository: https://github.com/aidinhamedi/optimizer-benchmark

## Overview

##### Project Type & Purpose
- This is an open-source benchmarking suite (Python CLI tool) for evaluating and comparing PyTorch optimizers on 2D test functions, producing both raw results and a static website of rankings and trajectory visualizations.

##### High-Level Architecture Components
a. **CLI Orchestrator**
- runner.py: entry point that reads config.toml, kicks off benchmark runs.
- Makefile / uv (uv.sync): wrapper for installation & common commands.
b. **Benchmark Engine**
- benchmark/criterion.py: defines evaluation criteria.
- benchmark/evaluate.py: runs the optimization loops.
- benchmark/functions/: suite of 2D test functions.
- benchmark/utils/ (executor.py, model.py, surface.py): support code for running optimization, representing surfaces, logging.
c. **Hyperparameter Tuning**
- Optuna integration inside evaluate.py (search loops).
d. **Visualization Pipeline**
- benchmark/visualizer.py: generates trajectory plots.
- tools/doc_visualizations.py & tools/gen_comparison.py: produces the HTML snippets & summary tables for the docs site using Jinja templates.
e. **Result Storage**
- results/results.json: aggregated benchmark data.
- PNG/HTML plots under docs/vis or results subfolders.
f. **Documentation / Website**
- docs/: static website (HTML, CSS, JS) generated via Jinja templates, updated from tools/.
- GitHub Pages deployment (site lives at aidinhamedi.github.io/Optimizer-Benchmark).
g. **External Dependencies**
- PyTorch, pytorch_optimizer library, Optuna, Jinja2, toml parser, any plotting libs (e.g., Matplotlib or Plotly).

##### Relationships & Data Flow
1. **User** invokes `python runner.py` (or `make benchmark`)
2. Runner loads **config.toml →**
3. For each optimizer & function:
- **evaluate.py** uses **functions/** to compute gradients,
- **Optuna** for hyperparameter search,
- Records metrics via **criterion.py** →
- Writes raw data to **results/results.json**.
4. **benchmark/visualizer.py** reads results.json → generates per-optimizer & per-function trajectory plots (HTML/PNG) in docs/vis.
5. **tools/** scripts read results.json + plots → render summary pages via Jinja templates → place final HTML, CSS, JS under docs/.
6. **docs/** becomes the GitHub Pages site.

##### Architectural Patterns & Principles
- **Modular •** Separation of concerns: CLI, benchmarking, tuning, visualization, documentation.
- **Pipeline**: data flows from raw runs → results → visualizations → static site.
- **Extensibility**: easily plug in new functions or optimizers by adding files under benchmark/functions or adjusting config.toml.
- **Configuration-driven**: all parameters in config.toml.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration
    subgraph "Configuration"
        Config["config.toml"]:::cli
    end
    click Config "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/config.toml"

    %% CLI Orchestrator
    subgraph "CLI Orchestrator"
        Runner["runner.py"]:::cli
        Makefile["Makefile"]:::cli
        UV["uv.lock"]:::cli
    end
    click Runner "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/runner.py"
    click Makefile "https://github.com/aidinhamedi/optimizer-benchmark/tree/main/Makefile"
    click UV "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/uv.lock"

    Config -->|"loads"| Runner
    Runner -->|"invokes"| Evaluate

    %% Benchmark Engine
    subgraph "Benchmark Engine"
        criterion["benchmark/criterion.py"]:::compute
        evaluate["benchmark/evaluate.py"]:::compute
        click criterion "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/benchmark/criterion.py"
        click evaluate "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/benchmark/evaluate.py"

        subgraph "Functions Library"
            Functions["benchmark/functions/"]:::compute
        end
        click Functions "https://github.com/aidinhamedi/optimizer-benchmark/tree/main/benchmark/functions/"

        subgraph "Utilities"
            executor["executor.py"]:::compute
            model["model.py"]:::compute
            surface["surface.py"]:::compute
            click executor "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/benchmark/utils/executor.py"
            click model "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/benchmark/utils/model.py"
            click surface "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/benchmark/utils/surface.py"
        end
    end

    Runner -->|"runs benchmarks"| evaluate
    evaluate -->|"uses criteria"| criterion
    evaluate -->|"computes gradients"| Functions
    evaluate -->|"executes optimization"| executor
    evaluate -->|"builds models"| model
    evaluate -->|"samples surface"| surface

    %% External Dependencies
    subgraph "External Dependencies"
        PyTorch(["PyTorch"]):::external
        TorchOpt(["pytorch_optimizer"]):::external
        Optuna(["Optuna"]):::external
        TOML(["toml parser"]):::external
        PlotLib(["Matplotlib/Plotly"]):::external
        Jinja2(["Jinja2"]):::external
    end

    Runner -->|"parses config"| TOML
    evaluate -->|"hyperparameter search"| Optuna
    evaluate -->|"uses deep learning"| PyTorch
    evaluate -->|"optimizer implementations"| TorchOpt

    %% Result Storage
    results["results/results.json"]:::storage
    click results "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/results/results.json"
    evaluate -->|"raw metrics"| results

    %% Visualization Pipeline
    visualizer["benchmark/visualizer.py"]:::viz
    click visualizer "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/benchmark/visualizer.py"
    results -->|"reads metrics"| visualizer
    visualizer -->|"plots"| DocsVis

    %% Documentation Generators
    subgraph "Tools / Docs Generator"
        docVis["tools/doc_visualizations.py"]:::viz
        genComp["tools/gen_comparison.py"]:::viz
        click docVis "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/tools/doc_visualizations.py"
        click genComp "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/tools/gen_comparison.py"

        subgraph "Jinja Templates"
            card["card.html"]:::viz
            page["page.html"]:::viz
            scriptT["script.js"]:::viz
            stylesT["styles.css"]:::viz
            click card "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/tools/templates/card.html"
            click page "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/tools/templates/page.html"
            click scriptT "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/tools/templates/script.js"
            click stylesT "https://github.com/aidinhamedi/optimizer-benchmark/blob/main/tools/templates/styles.css"
        end
    end

    results -->|"data input"| docVis
    DocsVis -->|"visuals input"| docVis
    docVis -->|"renders pages"| DocsSite
    genComp -->|"renders comparison"| DocsSite
    Jinja2 -.->|"templates"| docVis
    Jinja2 -.-> genComp

    %% Documentation Site
    subgraph "Documentation / Website"
        DocsVis["docs/vis/"]:::docs
        DocsSite["docs/"]:::docs
        click DocsVis "https://github.com/aidinhamedi/optimizer-benchmark/tree/main/docs/vis/"
        click DocsSite "https://github.com/aidinhamedi/optimizer-benchmark/tree/main/docs/"
    end

    %% Styles
    classDef cli fill:#D0E8FF,stroke:#0366d6,stroke-width:1px
    classDef compute fill:#E2F0D9,stroke:#28a745,stroke-width:1px
    classDef storage fill:#FFE8A0,stroke:#d47d00,stroke-width:1px
    classDef viz fill:#F3E5FF,stroke:#6f42c1,stroke-width:1px
    classDef docs fill:#F5E0E0,stroke:#d6336c,stroke-width:1px
    classDef external fill:#EDEDED,stroke:#6c757d,stroke-width:1px

    class Runner,Makefile,UV,Config cli
    class evaluate,criterion,Functions,executor,model,surface compute
    class results storage
    class visualizer,docVis,genComp,card,page,scriptT,stylesT viz
    class DocsVis,DocsSite docs
    class PyTorch,TorchOpt,Optuna,TOML,PlotLib,Jinja2 external
```
