# quettaflop/quettabench architecture

> Architecture diagram of the GitHub repository quettaflop/quettabench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/quettaflop/quettabench
- Repository: https://github.com/quettaflop/quettabench
- Video tour (about a minute): https://gitdiagram.com/quettaflop/quettabench/video

## Overview

QuettaBench benchmarks LLM inference with production-oriented single-turn, multi-turn, and stress workloads, measuring request latency and token timing against OpenAI-compatible servers. Its workload modules provide prompt/session data and distributional replay; engine adapters provide chat and completions interfaces. The README also describes opt-in GPU/live-server profiling and QuettaServe-versus-vLLM cross-validation. The listed tree does not include the documented benchmark runner or launch scripts, so their orchestration and result-writing links are omitted; profiling relationships are based on README descriptions and the sampled probe/emitter excerpts.

Main entry points: [`src/workloads/profiles.py`](https://github.com/quettaflop/quettabench/blob/main/src/workloads/profiles.py) (Workload profiles), [`src/workloads/distributional.py`](https://github.com/quettaflop/quettabench/blob/main/src/workloads/distributional.py) (Synthetic replay), [`profiling/probes`](https://github.com/quettaflop/quettabench/tree/main/profiling/probes) (Live-server probes), [`src/modes/multi_turn.py`](https://github.com/quettaflop/quettabench/blob/main/src/modes/multi_turn.py) (Multi-turn mode)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workloads["Workload modeling"]
  node_profiles["Workload profiles<br/>[profiles.py]"]
  node_datasets["Prompt datasets<br/>[dataset.py]"]
  node_arrival["Arrival patterns<br/>[arrival.py]"]
  node_trace_dist["Trace distributions"]
  node_distributional["Synthetic replay<br/>[distributional.py]"]
  node_mooncake["Mooncake workloads<br/>[mooncake.py]"]
end

subgraph group_execution["Benchmark interfaces"]
  node_single_mode["Single-turn mode<br/>[single_turn.py]"]
  node_multi_mode["Multi-turn mode<br/>[multi_turn.py]"]
  node_chat_engine["Chat adapter<br/>[openai_chat.py]"]
  node_completion_engine["Completions adapter"]
end

subgraph group_profiling["Profiling tools"]
  node_kernel_probes["GPU kernel probes"]
  node_live_probes["Live-server probes"]
  node_emitters["Profile emitters"]
end

subgraph group_validation["Cross-validation"]
  node_crossval["Cross-validation tools"]
end

node_user(("Benchmark user"))
node_inference_server["Inference server"]
node_profile_data["Profile measurements"]
node_baseline["Cached vLLM baseline"]

node_user -->|"selects mode"| node_single_mode
node_user -->|"selects mode"| node_multi_mode
node_single_mode -->|"filters profiles"| node_profiles
node_multi_mode -->|"uses profiles"| node_profiles
node_profiles -->|"selects prompts"| node_datasets
node_profiles -->|"selects replay"| node_distributional
node_distributional -->|"samples traces"| node_trace_dist
node_distributional -.->|"supports replay"| node_mooncake
node_arrival -.->|"paces turns"| node_multi_mode
node_chat_engine -->|"streams requests"| node_inference_server
node_completion_engine -->|"streams requests"| node_inference_server
node_user -->|"runs probes"| node_kernel_probes
node_user -->|"runs probes"| node_live_probes
node_kernel_probes -->|"writes measurements"| node_profile_data
node_live_probes -->|"probes server"| node_inference_server
node_live_probes -->|"writes measurements"| node_profile_data
node_emitters -->|"curates measurements"| node_profile_data
node_crossval -->|"compares against"| node_baseline

click node_single_mode "https://github.com/quettaflop/quettabench/blob/main/src/modes/single_turn.py"
click node_multi_mode "https://github.com/quettaflop/quettabench/blob/main/src/modes/multi_turn.py"
click node_profiles "https://github.com/quettaflop/quettabench/blob/main/src/workloads/profiles.py"
click node_datasets "https://github.com/quettaflop/quettabench/blob/main/src/workloads/dataset.py"
click node_arrival "https://github.com/quettaflop/quettabench/blob/main/src/workloads/arrival.py"
click node_trace_dist "https://github.com/quettaflop/quettabench/blob/main/src/workloads/trace_distributions.py"
click node_distributional "https://github.com/quettaflop/quettabench/blob/main/src/workloads/distributional.py"
click node_mooncake "https://github.com/quettaflop/quettabench/blob/main/src/workloads/mooncake.py"
click node_chat_engine "https://github.com/quettaflop/quettabench/blob/main/src/engines/openai_chat.py"
click node_completion_engine "https://github.com/quettaflop/quettabench/blob/main/src/engines/openai_completions.py"
click node_kernel_probes "https://github.com/quettaflop/quettabench/tree/main/profiling/kernel_composed/probes"
click node_live_probes "https://github.com/quettaflop/quettabench/tree/main/profiling/probes"
click node_emitters "https://github.com/quettaflop/quettabench/tree/main/profiling/emit"
click node_crossval "https://github.com/quettaflop/quettabench/tree/main/scripts/crossval"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_profiles,node_datasets,node_arrival,node_trace_dist,node_distributional,node_mooncake,node_user toneBlue
class node_single_mode,node_multi_mode,node_chat_engine,node_completion_engine,node_baseline toneAmber
class node_kernel_probes,node_live_probes,node_emitters,node_inference_server toneMint
class node_crossval toneRose
class node_profile_data toneIndigo
```

## Components

### Workload modeling

- [Workload profiles](https://github.com/quettaflop/quettabench/blob/main/src/workloads/profiles.py): `src/workloads/profiles.py`
- [Prompt datasets](https://github.com/quettaflop/quettabench/blob/main/src/workloads/dataset.py): `src/workloads/dataset.py`
- [Arrival patterns](https://github.com/quettaflop/quettabench/blob/main/src/workloads/arrival.py): `src/workloads/arrival.py`
- [Trace distributions](https://github.com/quettaflop/quettabench/blob/main/src/workloads/trace_distributions.py): `src/workloads/trace_distributions.py`
- [Synthetic replay](https://github.com/quettaflop/quettabench/blob/main/src/workloads/distributional.py): `src/workloads/distributional.py`
- [Mooncake workloads](https://github.com/quettaflop/quettabench/blob/main/src/workloads/mooncake.py): `src/workloads/mooncake.py`

### Benchmark interfaces

- [Single-turn mode](https://github.com/quettaflop/quettabench/blob/main/src/modes/single_turn.py): `src/modes/single_turn.py`
- [Multi-turn mode](https://github.com/quettaflop/quettabench/blob/main/src/modes/multi_turn.py): `src/modes/multi_turn.py`
- [Chat adapter](https://github.com/quettaflop/quettabench/blob/main/src/engines/openai_chat.py): `src/engines/openai_chat.py`
- [Completions adapter](https://github.com/quettaflop/quettabench/blob/main/src/engines/openai_completions.py): `src/engines/openai_completions.py`

### Profiling tools

- [GPU kernel probes](https://github.com/quettaflop/quettabench/tree/main/profiling/kernel_composed/probes): `profiling/kernel_composed/probes`
- [Live-server probes](https://github.com/quettaflop/quettabench/tree/main/profiling/probes): `profiling/probes`
- [Profile emitters](https://github.com/quettaflop/quettabench/tree/main/profiling/emit): `profiling/emit`

### Cross-validation

- [Cross-validation tools](https://github.com/quettaflop/quettabench/tree/main/scripts/crossval): `scripts/crossval`

### Other components

- **Benchmark user**
- **Inference server**
- **Profile measurements**
- **Cached vLLM baseline**

## Connections

- Benchmark user → Single-turn mode: selects mode
- Benchmark user → Multi-turn mode: selects mode
- Single-turn mode → Workload profiles: filters profiles
- Multi-turn mode → Workload profiles: uses profiles
- Workload profiles → Prompt datasets: selects prompts
- Workload profiles → Synthetic replay: selects replay
- Synthetic replay → Trace distributions: samples traces
- Synthetic replay → Mooncake workloads: supports replay
- Arrival patterns → Multi-turn mode: paces turns
- Chat adapter → Inference server: streams requests
- Completions adapter → Inference server: streams requests
- Benchmark user → GPU kernel probes: runs probes
- Benchmark user → Live-server probes: runs probes
- GPU kernel probes → Profile measurements: writes measurements
- Live-server probes → Inference server: probes server
- Live-server probes → Profile measurements: writes measurements
- Profile emitters → Profile measurements: curates measurements
- Cross-validation tools → Cached vLLM baseline: compares against