# aidinhamedi/ml-optimizer-benchmark architecture

> Architecture diagram of the GitHub repository aidinhamedi/ml-optimizer-benchmark, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aidinhamedi/ml-optimizer-benchmark
- Repository: https://github.com/aidinhamedi/ml-optimizer-benchmark

## Overview

##### Project type and purpose
- This is a **command-line benchmarking suite** (a Python “tool”) for evaluating PyTorch optimizers on 2D test functions.
- It loads a TOML config (`config.toml`), runs hyperparameter tuning with Optuna, evaluates optimizer behavior on various functions in `benchmark/functions`, then visualizes the trajectories and writes results to JSON/plots.

##### File structure
- **Entry point**: `runner.py`
- **Core package**: `benchmark/`
- `hypertune.py` – orchestrates Optuna searches
- `evaluate.py` – evaluates one optimizer on one function
- `functions/` – definitions of 2D test functions (e.g. `ackley.py`, `rastrigin.py`, etc.)
- `utils/` – helpers:
– `executor.py` – runs the optimization loop on a given function + model
– `model.py` – wraps PyTorch modules & optimizer from `pytorch_optimizer`
– `surface.py` – generates grid data for contour plots
- **Visualization**: `benchmark/visualizer.py`
- **Config**: `config.toml`
- **Results folder**: `results/` (JSON + images)
- **Docs** and **tools/** for generating/ syncing documentation

##### Extract call-flow from README & filenames
- `runner.py` loads config → calls the hyperparameter‐tuning driver → calls evaluation → calls visualization → writes to `results/`.

##### Main components
A. **Runner (CLI)**
- Function: `main()` in `runner.py`
- Calls:
– `ConfigLoader.load(config.toml)`
– `BenchmarkTuner.run_all()` (in `benchmark/hypertune.py`)
– `Visualizer.generate_all()` (in `benchmark/visualizer.py`)
B. **BenchmarkTuner** (in `benchmark/hypertune.py`)
- Method: `run_all()` iterates optimizers & functions
- Calls per‐optimizer: `run_hyperparameter_search(optimizer_name)`
- Under the hood uses Optuna’s `study.optimize()` which calls `objective(trial)`
C. **Evaluator** (in `benchmark/evaluate.py`)
- Function: `evaluate_one(optimizer, function, hyperparams)`
- Calls:
– `Executor.execute(model, func, iterations)`
D. **Executor** (in `benchmark/utils/executor.py`)
- Class: `Executor`
- Method: `execute()`
– Instantiates `Model` (from `model.py`) with a PyTorch network + chosen optimizer
– Loops: compute gradient on `func(point)` → update parameters → record trajectory
E. **Model** (in `benchmark/utils/model.py`)
- Wraps PyTorch module + `pytorch_optimizer` call
F. **Surface** (in `benchmark/utils/surface.py`)
- Generates 2D grid of loss values for contour plots
G. **Visualizer** (in `benchmark/visualizer.py`)
- Function: `generate_plots(results.json, trajectories)`
- Renders contour + overlayed optimizer paths → saves PNG/HTML under `results/vis/`
H. **Outputs**
- `results/results.json`
- `results/vis/<optimizer>.png or .html`

##### Relationships & interactions
- **Data flow**:
config.toml → Runner → BenchmarkTuner → Optuna trials → Evaluator → Executor → Model + Function → recorded trajectories → Visualizer → results/
Runner → BenchmarkTuner → Evaluator → Executor → Model & Function
Runner → Visualizer
- **External services**:
– PyTorch + pytorch_optimizer
– Optuna

##### Key architectural patterns & principles
- **Modular layering**:
– CLI layer (runner.py)
– Orchestration layer (hypertune.py)
– Evaluation layer (evaluate.py + utils)
– Visualization layer (visualizer.py)
- **Separation of concerns**: config, benchmarking logic, function definitions, plotting
- **Plugin-style function repository**: any `.py` in `benchmark/functions/` is automatically picked up

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Layer
    subgraph "CLI Layer"
        runner["runner.py\nmain()"]:::orchestration
        config["config.toml"]:::data
    end

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        tuner["benchmark/hypertune.py\nBenchmarkTuner.run_all()"]:::orchestration
        optuna["Optuna\nstudy.optimize()"]:::external
    end

    %% Evaluation Layer
    subgraph "Evaluation Layer"
        evaluate["benchmark/evaluate.py\nevaluate_one()"]:::evaluation
        executor["benchmark/utils/executor.py\nExecutor.execute()"]:::evaluation
        model["benchmark/utils/model.py\nModel"]:::evaluation
        subgraph "Function Definitions"
            ackley["ackley.py"]:::evaluation
            rastrigin["rastrigin.py"]:::evaluation
        end
    end

    %% Visualization Layer
    subgraph "Visualization Layer"
        surface["benchmark/utils/surface.py\nSurface.generate()"]:::visualization
        visualizer["benchmark/visualizer.py\ngenerate_plots()"]:::visualization
    end

    %% Persistence Layer
    subgraph "Persistence Layer"
        results_json["results/results.json"]:::data
        results_vis["results/vis/"]:::data
    end

    %% Flow
    config -->|loads| runner
    runner -->|calls| tuner
    runner -->|calls| visualizer
    tuner -->|uses| optuna
    tuner -->|invokes| evaluate
    optuna -->|triggers| evaluate
    evaluate -->|calls| executor
    executor -->|instantiates| model
    executor -->|evaluates| ackley
    executor -->|evaluates| rastrigin
    evaluate -->|writes| results_json
    results_json -->|input to| visualizer
    visualizer -->|generates grid| surface
    visualizer -->|outputs| results_vis

    %% Click Events
    click runner "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/runner.py"
    click config "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/config.toml"
    click tuner "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/benchmark/hypertune.py"
    click evaluate "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/benchmark/evaluate.py"
    click executor "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/benchmark/utils/executor.py"
    click model "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/benchmark/utils/model.py"
    click surface "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/benchmark/utils/surface.py"
    click visualizer "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/benchmark/visualizer.py"
    click ackley "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/benchmark/functions/ackley.py"
    click rastrigin "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/benchmark/functions/rastrigin.py"
    click results_json "https://github.com/aidinhamedi/ml-optimizer-benchmark/blob/main/results/results.json"
    click results_vis "https://github.com/aidinhamedi/ml-optimizer-benchmark/tree/main/results/vis/"

    %% Styles
    classDef orchestration fill:#CFE2F3,stroke:#064273,color:#064273
    classDef evaluation fill:#D9EAD3,stroke:#38761D,color:#38761D
    classDef visualization fill:#FCE5CD,stroke:#E69138,color:#E69138
    classDef data fill:#EDEDED,stroke:#AAAAAA,color:#333333
    classDef external fill:#EAD1DC,stroke:#C00000,color:#C00000
```
