# windmill-labs/windmill-benchmarks architecture

> Architecture diagram of the GitHub repository windmill-labs/windmill-benchmarks, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/windmill-labs/windmill-benchmarks
- Repository: https://github.com/windmill-labs/windmill-benchmarks

## Overview

This repository is a benchmark suite: it compares workflow orchestrators and separately benchmarks TPC-H data integrations. The main user workflow is to run platform-specific benchmarks, then inspect or aggregate results; the pipeline branch runs analytical queries over datasets and writes outputs. Sampled code confirms some Hatchet and Temporal wiring and Airflow’s S3 data flow. Other platform implementations are represented by their tree paths, but their internal wiring is not inferred from filenames.

Main entry points: [`competitors/2026-09/hatchet/app/run_bench.py`](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/hatchet/app/run_bench.py) (Hatchet runner), [`competitors/temporal/flows/go/cmd/starter/main.go`](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/temporal/flows/go/cmd/starter/main.go) (Temporal starter), [`competitors/2026-09/hatchet/app/worker.py`](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/hatchet/app/worker.py) (Hatchet worker), [`competitors/temporal/flows/go/worker/worker.go`](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/temporal/flows/go/worker/worker.go) (Temporal worker)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_comparison["Orchestrator comparisons"]
  node_windmill_bench["Windmill benchmark<br/>[wm_bench.py]"]
  node_windmill_dedicated["Windmill dedicated"]
  node_airflow_bench["Airflow benchmark<br/>[airflow_harness.py]"]
  node_hatchet_run["Hatchet runner<br/>[run_bench.py]"]
  node_hatchet_defs["Hatchet definitions<br/>[bench_defs.py]"]
  node_hatchet_worker["Hatchet worker<br/>[worker.py]"]
  node_kestra_bench["Kestra benchmark<br/>[kestra_bench.py]"]
  node_prefect_bench["Prefect benchmark<br/>[run_bench.py]"]
  node_temporal_starter["Temporal starter<br/>[main.go]"]
  node_temporal_worker["Temporal worker<br/>[worker.go]"]
  node_temporal_typescript["Temporal TypeScript<br/>[worker.ts]"]
end

subgraph group_analysis["Results analysis"]
  node_result_aggregate["Result aggregation<br/>[aggregate.py]"]
  node_result_validate["Result validation<br/>[validate.py]"]
  node_result_analysis["Competitor analysis"]
end

subgraph group_pipelines["Data integrations"]
  node_airflow_tpc["Airflow TPC-H<br/>[tpc_h.py]"]
  node_duckdb_tpc["DuckDB TPC-H<br/>[tpc_h.py]"]
  node_polars_tpc["Polars TPC-H<br/>[tpc_h.py]"]
  node_spark_tpc["Spark TPC-H<br/>[tpc_h.py]"]
end

node_user(("Benchmark user"))
node_orchestrator_services{{"Orchestrator services"}}
node_pipeline_user(("Pipeline user"))
node_s3[("S3 storage")]

node_user -->|"runs"| node_windmill_bench
node_user -->|"runs"| node_windmill_dedicated
node_user -->|"runs"| node_airflow_bench
node_user -->|"runs"| node_hatchet_run
node_user -->|"runs"| node_kestra_bench
node_user -->|"runs"| node_prefect_bench
node_user -->|"runs"| node_temporal_starter
node_hatchet_run -->|"invokes"| node_hatchet_defs
node_hatchet_worker -->|"registers"| node_hatchet_defs
node_hatchet_run -->|"submits runs"| node_orchestrator_services
node_hatchet_worker -->|"polls tasks"| node_orchestrator_services
node_temporal_starter -->|"launches process"| node_temporal_worker
node_temporal_starter -->|"starts workflow"| node_orchestrator_services
node_temporal_worker -->|"polls tasks"| node_orchestrator_services
node_temporal_typescript -->|"starts workflow"| node_orchestrator_services
node_user -->|"aggregates results"| node_result_aggregate
node_user -->|"validates results"| node_result_validate
node_user -->|"analyzes results"| node_result_analysis
node_pipeline_user -->|"runs queries"| node_airflow_tpc
node_pipeline_user -->|"runs queries"| node_duckdb_tpc
node_pipeline_user -->|"runs queries"| node_polars_tpc
node_pipeline_user -->|"runs queries"| node_spark_tpc
node_airflow_tpc -->|"reads and writes"| node_s3

click node_windmill_bench "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/windmill/wm_bench.py"
click node_windmill_dedicated "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/windmill/wm_dedicated_bench.py"
click node_airflow_bench "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/airflow/airflow_harness.py"
click node_hatchet_run "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/hatchet/app/run_bench.py"
click node_hatchet_defs "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/hatchet/app/bench_defs.py"
click node_hatchet_worker "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/hatchet/app/worker.py"
click node_kestra_bench "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/kestra/kestra_bench.py"
click node_prefect_bench "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/prefect/run_bench.py"
click node_temporal_starter "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/temporal/flows/go/cmd/starter/main.go"
click node_temporal_worker "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/temporal/flows/go/worker/worker.go"
click node_temporal_typescript "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/temporal/flows/typescript/bench_10/src/worker.ts"
click node_result_aggregate "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/aggregate.py"
click node_result_validate "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/validate.py"
click node_result_analysis "https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/hatchet/benchmark_analyzer.py"
click node_airflow_tpc "https://github.com/windmill-labs/windmill-benchmarks/blob/main/pipelines/airflow/tpc_h.py"
click node_duckdb_tpc "https://github.com/windmill-labs/windmill-benchmarks/blob/main/pipelines/duckdb/tpc_h.py"
click node_polars_tpc "https://github.com/windmill-labs/windmill-benchmarks/blob/main/pipelines/polars/tpc_h.py"
click node_spark_tpc "https://github.com/windmill-labs/windmill-benchmarks/blob/main/pipelines/spark/tpc_h.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_windmill_bench,node_windmill_dedicated,node_airflow_bench,node_hatchet_run,node_hatchet_defs,node_hatchet_worker,node_kestra_bench,node_prefect_bench,node_temporal_starter,node_temporal_worker,node_temporal_typescript,node_user,node_pipeline_user toneBlue
class node_result_aggregate,node_result_validate,node_result_analysis,node_s3 toneAmber
class node_airflow_tpc,node_duckdb_tpc,node_polars_tpc,node_spark_tpc toneMint
class node_orchestrator_services toneIndigo
```

## Components

### Orchestrator comparisons

- [Windmill benchmark](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/windmill/wm_bench.py): `competitors/2026-09/windmill/wm_bench.py`
- [Windmill dedicated](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/windmill/wm_dedicated_bench.py): `competitors/2026-09/windmill/wm_dedicated_bench.py`
- [Airflow benchmark](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/airflow/airflow_harness.py): `competitors/2026-09/airflow/airflow_harness.py`
- [Hatchet runner](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/hatchet/app/run_bench.py): `competitors/2026-09/hatchet/app/run_bench.py`
- [Hatchet definitions](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/hatchet/app/bench_defs.py): `competitors/2026-09/hatchet/app/bench_defs.py`
- [Hatchet worker](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/hatchet/app/worker.py): `competitors/2026-09/hatchet/app/worker.py`
- [Kestra benchmark](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/kestra/kestra_bench.py): `competitors/2026-09/kestra/kestra_bench.py`
- [Prefect benchmark](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/prefect/run_bench.py): `competitors/2026-09/prefect/run_bench.py`
- [Temporal starter](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/temporal/flows/go/cmd/starter/main.go): `competitors/temporal/flows/go/cmd/starter/main.go`
- [Temporal worker](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/temporal/flows/go/worker/worker.go): `competitors/temporal/flows/go/worker/worker.go`
- [Temporal TypeScript](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/temporal/flows/typescript/bench_10/src/worker.ts): `competitors/temporal/flows/typescript/bench_10/src/worker.ts`

### Results analysis

- [Result aggregation](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/aggregate.py): `competitors/2026-09/aggregate.py`
- [Result validation](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/2026-09/validate.py): `competitors/2026-09/validate.py`
- [Competitor analysis](https://github.com/windmill-labs/windmill-benchmarks/blob/main/competitors/hatchet/benchmark_analyzer.py): `competitors/hatchet/benchmark_analyzer.py`

### Data integrations

- [Airflow TPC-H](https://github.com/windmill-labs/windmill-benchmarks/blob/main/pipelines/airflow/tpc_h.py): `pipelines/airflow/tpc_h.py`
- [DuckDB TPC-H](https://github.com/windmill-labs/windmill-benchmarks/blob/main/pipelines/duckdb/tpc_h.py): `pipelines/duckdb/tpc_h.py`
- [Polars TPC-H](https://github.com/windmill-labs/windmill-benchmarks/blob/main/pipelines/polars/tpc_h.py): `pipelines/polars/tpc_h.py`
- [Spark TPC-H](https://github.com/windmill-labs/windmill-benchmarks/blob/main/pipelines/spark/tpc_h.py): `pipelines/spark/tpc_h.py`

### Other components

- **Benchmark user**
- **Orchestrator services**
- **Pipeline user**
- **S3 storage**

## Connections

- Benchmark user → Windmill benchmark: runs
- Benchmark user → Windmill dedicated: runs
- Benchmark user → Airflow benchmark: runs
- Benchmark user → Hatchet runner: runs
- Benchmark user → Kestra benchmark: runs
- Benchmark user → Prefect benchmark: runs
- Benchmark user → Temporal starter: runs
- Hatchet runner → Hatchet definitions: invokes
- Hatchet worker → Hatchet definitions: registers
- Hatchet runner → Orchestrator services: submits runs
- Hatchet worker → Orchestrator services: polls tasks
- Temporal starter → Temporal worker: launches process
- Temporal starter → Orchestrator services: starts workflow
- Temporal worker → Orchestrator services: polls tasks
- Temporal TypeScript → Orchestrator services: starts workflow
- Benchmark user → Result aggregation: aggregates results
- Benchmark user → Result validation: validates results
- Benchmark user → Competitor analysis: analyzes results
- Pipeline user → Airflow TPC-H: runs queries
- Pipeline user → DuckDB TPC-H: runs queries
- Pipeline user → Polars TPC-H: runs queries
- Pipeline user → Spark TPC-H: runs queries
- Airflow TPC-H → S3 storage: reads and writes