# minio/warp architecture

> Architecture diagram of the GitHub repository minio/warp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/minio/warp
- Repository: https://github.com/minio/warp

## Overview

Warp is an S3 benchmarking tool with CLI-driven object workloads, distributed client execution, live/final result analysis, and Iceberg catalog/table benchmarks. The usual flow configures a benchmark, executes it against S3 (locally or through clients), then aggregates and presents results. The graph includes documented benchmark branches and sampled internal layers; unsampled command implementations are represented only at their exact paths, and their specific call paths are not inferred.

Main entry points: [`pkg/aggregate/aggregate.go`](https://github.com/minio/warp/blob/master/pkg/aggregate/aggregate.go) (Result aggregation), [`cli/cli.go`](https://github.com/minio/warp/blob/master/cli/cli.go) (Command interface), [`cli`](https://github.com/minio/warp/tree/master/cli) (S3 workloads), [`cli/benchserver.go`](https://github.com/minio/warp/blob/master/cli/benchserver.go) (Run coordinator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_commands["CLI workflows"]
  node_entry["Program entry<br/>[main.go]"]
  node_cli["Command interface<br/>[cli.go]"]
  node_config["Flags and config<br/>[flags.go]"]
end

subgraph group_execution["Benchmark execution"]
  node_s3workloads["S3 workloads"]
  node_clientsetup["S3 client setup<br/>[client.go]"]
  node_transport["HTTP transports"]
  node_generator["Data generation<br/>[generator.go]"]
end

subgraph group_distributed["Distributed runs"]
  node_benchclient["Client protocol<br/>[benchclient.go]"]
  node_benchserver["Run coordinator<br/>[benchserver.go]"]
end

subgraph group_results["Results and analysis"]
  node_aggregate["Result aggregation<br/>[aggregate.go]"]
  node_live["Live updates<br/>[live.go]"]
  node_requests["Request statistics<br/>[requests.go]"]
  node_throughput["Throughput metrics<br/>[throughput.go]"]
  node_analyze["Analysis commands<br/>[analyze.go]"]
end

subgraph group_iceberg["Iceberg benchmarks"]
  node_icebergcommands["Iceberg workloads"]
  node_catalog["Catalog operations<br/>[catalog.go]"]
  node_commit["Table commits<br/>[commit.go]"]
  node_dataset["Dataset generation<br/>[dataset.go]"]
  node_parquet["Parquet encoding<br/>[parquet.go]"]
end

subgraph group_presentation["Result presentation"]
  node_monitor["Monitor API<br/>[api.go]"]
  node_webui["Live web UI<br/>[server.go]"]
  node_handlers["UI data handler<br/>[handlers.go]"]
end

node_user(("Operator"))
node_s3{{"S3 service"}}
node_remoteclients(("Warp clients"))
node_browser(("Browser"))

node_user -->|"runs command"| node_entry
node_entry -->|"starts CLI"| node_cli
node_user -->|"selects benchmark"| node_cli
node_cli -->|"uses options"| node_config
node_cli -.->|"dispatches workload"| node_s3workloads
node_s3workloads -.->|"uses S3 client"| node_clientsetup
node_clientsetup -->|"builds transport"| node_transport
node_clientsetup -->|"connects to"| node_s3
node_s3workloads -.->|"uses generated data"| node_generator
node_cli -->|"selects Iceberg run"| node_icebergcommands
node_icebergcommands -->|"runs catalog operations"| node_catalog
node_icebergcommands -->|"generates commits"| node_commit
node_icebergcommands -->|"writes table data"| node_dataset
node_dataset -.->|"uses encoding"| node_parquet
node_user -->|"starts distributed run"| node_benchserver
node_benchserver -->|"coordinates clients"| node_remoteclients
node_remoteclients -->|"accepts run requests"| node_benchclient
node_benchclient -->|"executes benchmark"| node_s3workloads
node_benchserver -->|"merges results"| node_aggregate
node_s3workloads -.->|"records operations"| node_aggregate
node_aggregate -->|"updates live data"| node_live
node_aggregate -->|"computes request stats"| node_requests
node_aggregate -->|"computes throughput"| node_throughput
node_user -->|"requests analysis"| node_analyze
node_analyze -->|"reads results"| node_aggregate
node_benchserver -->|"serves run status"| node_monitor
node_monitor -->|"requests updates"| node_live
node_webui -->|"registers routes"| node_handlers
node_handlers -->|"polls live data"| node_live
node_browser -.->|"views results"| node_webui

click node_entry "https://github.com/minio/warp/blob/master/main.go"
click node_cli "https://github.com/minio/warp/blob/master/cli/cli.go"
click node_config "https://github.com/minio/warp/blob/master/cli/flags.go"
click node_s3workloads "https://github.com/minio/warp/tree/master/cli"
click node_clientsetup "https://github.com/minio/warp/blob/master/cli/client.go"
click node_transport "https://github.com/minio/warp/blob/master/cli/client_transport.go"
click node_generator "https://github.com/minio/warp/blob/master/pkg/generator/generator.go"
click node_icebergcommands "https://github.com/minio/warp/tree/master/cli"
click node_catalog "https://github.com/minio/warp/blob/master/pkg/iceberg/catalog.go"
click node_commit "https://github.com/minio/warp/blob/master/pkg/iceberg/commit.go"
click node_dataset "https://github.com/minio/warp/blob/master/pkg/iceberg/dataset.go"
click node_parquet "https://github.com/minio/warp/blob/master/pkg/iceberg/parquet.go"
click node_benchclient "https://github.com/minio/warp/blob/master/cli/benchclient.go"
click node_benchserver "https://github.com/minio/warp/blob/master/cli/benchserver.go"
click node_aggregate "https://github.com/minio/warp/blob/master/pkg/aggregate/aggregate.go"
click node_live "https://github.com/minio/warp/blob/master/pkg/aggregate/live.go"
click node_requests "https://github.com/minio/warp/blob/master/pkg/aggregate/requests.go"
click node_throughput "https://github.com/minio/warp/blob/master/pkg/aggregate/throughput.go"
click node_analyze "https://github.com/minio/warp/blob/master/cli/analyze.go"
click node_monitor "https://github.com/minio/warp/blob/master/api/api.go"
click node_webui "https://github.com/minio/warp/blob/master/wui/server.go"
click node_handlers "https://github.com/minio/warp/blob/master/wui/handlers.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_cli,node_config,node_remoteclients,node_browser toneBlue
class node_s3workloads,node_clientsetup,node_transport,node_generator toneAmber
class node_benchclient,node_benchserver toneMint
class node_aggregate,node_live,node_requests,node_throughput,node_analyze toneRose
class node_icebergcommands,node_catalog,node_commit,node_dataset,node_parquet,node_user,node_s3 toneIndigo
class node_monitor,node_webui,node_handlers toneTeal
```

## Components

### CLI workflows

- [Program entry](https://github.com/minio/warp/blob/master/main.go): `main.go`
- [Command interface](https://github.com/minio/warp/blob/master/cli/cli.go): `cli/cli.go`
- [Flags and config](https://github.com/minio/warp/blob/master/cli/flags.go): `cli/flags.go`

### Benchmark execution

- [S3 workloads](https://github.com/minio/warp/tree/master/cli): `cli`
- [S3 client setup](https://github.com/minio/warp/blob/master/cli/client.go): `cli/client.go`
- [HTTP transports](https://github.com/minio/warp/blob/master/cli/client_transport.go): `cli/client_transport.go`
- [Data generation](https://github.com/minio/warp/blob/master/pkg/generator/generator.go): `pkg/generator/generator.go`

### Distributed runs

- [Client protocol](https://github.com/minio/warp/blob/master/cli/benchclient.go): `cli/benchclient.go`
- [Run coordinator](https://github.com/minio/warp/blob/master/cli/benchserver.go): `cli/benchserver.go`

### Results and analysis

- [Result aggregation](https://github.com/minio/warp/blob/master/pkg/aggregate/aggregate.go): `pkg/aggregate/aggregate.go`
- [Live updates](https://github.com/minio/warp/blob/master/pkg/aggregate/live.go): `pkg/aggregate/live.go`
- [Request statistics](https://github.com/minio/warp/blob/master/pkg/aggregate/requests.go): `pkg/aggregate/requests.go`
- [Throughput metrics](https://github.com/minio/warp/blob/master/pkg/aggregate/throughput.go): `pkg/aggregate/throughput.go`
- [Analysis commands](https://github.com/minio/warp/blob/master/cli/analyze.go): `cli/analyze.go`

### Iceberg benchmarks

- [Iceberg workloads](https://github.com/minio/warp/tree/master/cli): `cli`
- [Catalog operations](https://github.com/minio/warp/blob/master/pkg/iceberg/catalog.go): `pkg/iceberg/catalog.go`
- [Table commits](https://github.com/minio/warp/blob/master/pkg/iceberg/commit.go): `pkg/iceberg/commit.go`
- [Dataset generation](https://github.com/minio/warp/blob/master/pkg/iceberg/dataset.go): `pkg/iceberg/dataset.go`
- [Parquet encoding](https://github.com/minio/warp/blob/master/pkg/iceberg/parquet.go): `pkg/iceberg/parquet.go`

### Result presentation

- [Monitor API](https://github.com/minio/warp/blob/master/api/api.go): `api/api.go`
- [Live web UI](https://github.com/minio/warp/blob/master/wui/server.go): `wui/server.go`
- [UI data handler](https://github.com/minio/warp/blob/master/wui/handlers.go): `wui/handlers.go`

### Other components

- **Operator**
- **S3 service**
- **Warp clients**
- **Browser**

## Connections

- Operator → Program entry: runs command (evidence: `README.md`)
- Program entry → Command interface: starts CLI (evidence: `main.go`)
- Operator → Command interface: selects benchmark (evidence: `README.md`)
- Command interface → Flags and config: uses options (evidence: `cli/flags.go`)
- Command interface → S3 workloads: dispatches workload
- S3 workloads → S3 client setup: uses S3 client (evidence: `cli/client.go`)
- S3 client setup → HTTP transports: builds transport (evidence: `cli/client.go`)
- S3 client setup → S3 service: connects to (evidence: `README.md`)
- S3 workloads → Data generation: uses generated data
- Command interface → Iceberg workloads: selects Iceberg run (evidence: `README.md`)
- Iceberg workloads → Catalog operations: runs catalog operations (evidence: `cli/iceberg_catalog_read.go`)
- Iceberg workloads → Table commits: generates commits (evidence: `README.md`)
- Iceberg workloads → Dataset generation: writes table data (evidence: `cli/iceberg_sustained.go`)
- Dataset generation → Parquet encoding: uses encoding
- Operator → Run coordinator: starts distributed run (evidence: `README.md`)
- Run coordinator → Warp clients: coordinates clients (evidence: `README.md`)
- Warp clients → Client protocol: accepts run requests (evidence: `cli/benchclient.go`)
- Client protocol → S3 workloads: executes benchmark (evidence: `cli/benchclient.go`)
- Run coordinator → Result aggregation: merges results (evidence: `cli/benchserver.go`)
- S3 workloads → Result aggregation: records operations
- Result aggregation → Live updates: updates live data (evidence: `pkg/aggregate/aggregate.go`)
- Result aggregation → Request statistics: computes request stats (evidence: `pkg/aggregate/aggregate.go`)
- Result aggregation → Throughput metrics: computes throughput (evidence: `pkg/aggregate/aggregate.go`)
- Operator → Analysis commands: requests analysis (evidence: `README.md`)
- Analysis commands → Result aggregation: reads results (evidence: `cli/analyze.go`)
- Run coordinator → Monitor API: serves run status (evidence: `cli/benchserver.go`)
- Monitor API → Live updates: requests updates (evidence: `api/api.go`)
- Live web UI → UI data handler: registers routes (evidence: `wui/server.go`)
- UI data handler → Live updates: polls live data (evidence: `wui/handlers.go`)
- Browser → Live web UI: views results