# artificialray/cpu-kernel-baseline architecture

> Architecture diagram of the GitHub repository artificialray/cpu-kernel-baseline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/artificialray/cpu-kernel-baseline
- Repository: https://github.com/artificialray/cpu-kernel-baseline

## Overview

CPU-Kernel-Baseline evaluates agents that optimize AArch64 SIMD kernels. Its principal workflow selects benchmark definitions, provisions or reuses a target, starts an MCP kernel session, lets a harness compile and evaluate candidate code, and collects trajectories and results. The repository also supports a self-hosted litellm harness, kernel-definition and workload generation, and SIMD-loop benchmarks. Edges reflect README statements and source-index references or excerpts; several implementation details are unsampled, so uncertain connections are omitted.

Main entry points: [`test_scripts/bench_fleet.py`](https://github.com/artificialray/cpu-kernel-baseline/blob/main/test_scripts/bench_fleet.py) (Fleet driver), [`eval/evaluator.py`](https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/evaluator.py) (Litellm agent loop), [`skills/launch/launch_session.py`](https://github.com/artificialray/cpu-kernel-baseline/blob/main/skills/launch/launch_session.py) (Session launcher), [`mcp_app/agent_tools/base.py`](https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/agent_tools/base.py) (Kernel tool surface)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_run["Run orchestration"]
  node_fleet["Fleet driver<br/>[bench_fleet.py]"]
  node_adapters["Harness adapters"]
  node_launcher["Session launcher<br/>[launch_session.py]"]
  node_provision["Instance provisioning<br/>[provision.py]"]
  node_remote["Remote access<br/>[remote.py]"]
end

subgraph group_agent["Agent interface"]
  node_harness(("Agent harness"))
  node_ownloop["Litellm agent loop<br/>[evaluator.py]"]
  node_llm["LLM API dispatch<br/>[llm_call.py]"]
  node_providers["Model providers<br/>[llm_providers.py]"]
  node_mcpclient["MCP client bridge<br/>[mcp_client.py]"]
  node_server["MCP server<br/>[server.py]"]
  node_session["Session bootstrap<br/>[session.py]"]
  node_tools["Kernel tool surface<br/>[base.py]"]
  node_isa["ISA handling<br/>[isa.py]"]
end

subgraph group_bench["Kernel evaluation"]
  node_ops["Compile and evaluate<br/>[ops.py]"]
  node_contracts["Kernel contracts<br/>[contracts.py]"]
end

subgraph group_data["Workloads and kernels"]
  node_definitions["Model definitions"]
  node_workloads["Workload generation<br/>[gen_workload.py]"]
  node_problemset["Synthetic problem specs<br/>[problem.py]"]
  node_loopkernels["SIMD loop kernels<br/>[loop_215.c]"]
  node_loopcommon["SIMD loop primitives<br/>[loops.h]"]
end

subgraph group_analysis["Results and studies"]
  node_single["Single-shot evaluation<br/>[single_shot.py]"]
  node_wandb["Run logging<br/>[wandb_log_run.py]"]
  node_audits["Baseline audits"]
end

node_user(("Benchmark operator"))

node_user -->|"starts run"| node_fleet
node_fleet -->|"selects harness"| node_adapters
node_fleet -->|"prepares session"| node_launcher
node_launcher -->|"provisions instance"| node_provision
node_launcher -->|"uses remote access"| node_remote
node_adapters -->|"invokes"| node_harness
node_harness -->|"connects via MCP"| node_server
node_fleet -->|"drives own harness"| node_ownloop
node_ownloop -->|"requests completion"| node_llm
node_ownloop -->|"resolves provider"| node_providers
node_ownloop -->|"uses MCP tools"| node_mcpclient
node_mcpclient -->|"calls remotely"| node_server
node_server -->|"builds tools"| node_session
node_session -->|"constructs session"| node_tools
node_tools -->|"checks ISA"| node_isa
node_tools -->|"delegates operations"| node_ops
node_ops -->|"uses policy"| node_contracts
node_definitions -.->|"generates definitions"| node_workloads
node_problemset -.->|"benchmarks kernels"| node_loopkernels
node_loopkernels -->|"uses SIMD macros"| node_loopcommon
node_fleet -->|"logs runs"| node_wandb
node_audits -->|"audits baselines"| node_ops
node_fleet -.->|"supports single-shot"| node_single

click node_fleet "https://github.com/artificialray/cpu-kernel-baseline/blob/main/test_scripts/bench_fleet.py"
click node_adapters "https://github.com/artificialray/cpu-kernel-baseline/blob/main/test_scripts/harness_adapters.py"
click node_launcher "https://github.com/artificialray/cpu-kernel-baseline/blob/main/skills/launch/launch_session.py"
click node_provision "https://github.com/artificialray/cpu-kernel-baseline/blob/main/provisioning/provision.py"
click node_remote "https://github.com/artificialray/cpu-kernel-baseline/blob/main/provisioning/remote.py"
click node_ownloop "https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/evaluator.py"
click node_llm "https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/llm_call.py"
click node_providers "https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/llm_providers.py"
click node_mcpclient "https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/mcp_client.py"
click node_server "https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/server.py"
click node_session "https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/session.py"
click node_tools "https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/agent_tools/base.py"
click node_isa "https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/agent_tools/isa.py"
click node_ops "https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/agent_tools/ops.py"
click node_contracts "https://github.com/artificialray/cpu-kernel-baseline/blob/main/contracts.py"
click node_definitions "https://github.com/artificialray/cpu-kernel-baseline/blob/main/scripts/gen-definition/gen_model_definitions.py"
click node_workloads "https://github.com/artificialray/cpu-kernel-baseline/blob/main/scripts/gen-workload/gen_workload.py"
click node_problemset "https://github.com/artificialray/cpu-kernel-baseline/blob/main/dataset/problems/loop_012_particle_motion/problem.py"
click node_loopkernels "https://github.com/artificialray/cpu-kernel-baseline/blob/main/loops/loop_215.c"
click node_loopcommon "https://github.com/artificialray/cpu-kernel-baseline/blob/main/loops/common/loops.h"
click node_single "https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/single_shot.py"
click node_wandb "https://github.com/artificialray/cpu-kernel-baseline/blob/main/analysis/wandb_log_run.py"
click node_audits "https://github.com/artificialray/cpu-kernel-baseline/blob/main/analysis/audit_simd_baselines.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_fleet,node_adapters,node_launcher,node_provision,node_remote toneBlue
class node_harness,node_ownloop,node_llm,node_providers,node_mcpclient,node_server,node_session,node_tools,node_isa toneAmber
class node_ops,node_contracts toneMint
class node_definitions,node_workloads,node_problemset,node_loopkernels,node_loopcommon toneRose
class node_single,node_wandb,node_audits,node_user toneIndigo
```

## Components

### Run orchestration

- [Fleet driver](https://github.com/artificialray/cpu-kernel-baseline/blob/main/test_scripts/bench_fleet.py): `test_scripts/bench_fleet.py`
- [Harness adapters](https://github.com/artificialray/cpu-kernel-baseline/blob/main/test_scripts/harness_adapters.py): `test_scripts/harness_adapters.py`
- [Session launcher](https://github.com/artificialray/cpu-kernel-baseline/blob/main/skills/launch/launch_session.py): `skills/launch/launch_session.py`
- [Instance provisioning](https://github.com/artificialray/cpu-kernel-baseline/blob/main/provisioning/provision.py): `provisioning/provision.py`
- [Remote access](https://github.com/artificialray/cpu-kernel-baseline/blob/main/provisioning/remote.py): `provisioning/remote.py`

### Agent interface

- **Agent harness**
- [Litellm agent loop](https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/evaluator.py): `eval/evaluator.py`
- [LLM API dispatch](https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/llm_call.py): `eval/llm_call.py`
- [Model providers](https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/llm_providers.py): `eval/llm_providers.py`
- [MCP client bridge](https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/mcp_client.py): `eval/mcp_client.py`
- [MCP server](https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/server.py): `mcp_app/server.py`
- [Session bootstrap](https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/session.py): `mcp_app/session.py`
- [Kernel tool surface](https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/agent_tools/base.py): `mcp_app/agent_tools/base.py`
- [ISA handling](https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/agent_tools/isa.py): `mcp_app/agent_tools/isa.py`

### Kernel evaluation

- [Compile and evaluate](https://github.com/artificialray/cpu-kernel-baseline/blob/main/mcp_app/agent_tools/ops.py): `mcp_app/agent_tools/ops.py`
- [Kernel contracts](https://github.com/artificialray/cpu-kernel-baseline/blob/main/contracts.py): `contracts.py`

### Workloads and kernels

- [Model definitions](https://github.com/artificialray/cpu-kernel-baseline/blob/main/scripts/gen-definition/gen_model_definitions.py): `scripts/gen-definition/gen_model_definitions.py`
- [Workload generation](https://github.com/artificialray/cpu-kernel-baseline/blob/main/scripts/gen-workload/gen_workload.py): `scripts/gen-workload/gen_workload.py`
- [Synthetic problem specs](https://github.com/artificialray/cpu-kernel-baseline/blob/main/dataset/problems/loop_012_particle_motion/problem.py): `dataset/problems/loop_012_particle_motion/problem.py`
- [SIMD loop kernels](https://github.com/artificialray/cpu-kernel-baseline/blob/main/loops/loop_215.c): `loops/loop_215.c`
- [SIMD loop primitives](https://github.com/artificialray/cpu-kernel-baseline/blob/main/loops/common/loops.h): `loops/common/loops.h`

### Results and studies

- [Single-shot evaluation](https://github.com/artificialray/cpu-kernel-baseline/blob/main/eval/single_shot.py): `eval/single_shot.py`
- [Run logging](https://github.com/artificialray/cpu-kernel-baseline/blob/main/analysis/wandb_log_run.py): `analysis/wandb_log_run.py`
- [Baseline audits](https://github.com/artificialray/cpu-kernel-baseline/blob/main/analysis/audit_simd_baselines.py): `analysis/audit_simd_baselines.py`

### Other components

- **Benchmark operator**

## Connections

- Benchmark operator → Fleet driver: starts run (evidence: `README.md`)
- Fleet driver → Harness adapters: selects harness (evidence: `test_scripts/bench_fleet.py`)
- Fleet driver → Session launcher: prepares session (evidence: `test_scripts/bench_fleet.py`)
- Session launcher → Instance provisioning: provisions instance (evidence: `skills/launch/launch_session.py`)
- Session launcher → Remote access: uses remote access (evidence: `skills/launch/launch_session.py`)
- Harness adapters → Agent harness: invokes (evidence: `README.md`)
- Agent harness → MCP server: connects via MCP (evidence: `README.md`)
- Fleet driver → Litellm agent loop: drives own harness (evidence: `README.md`)
- Litellm agent loop → LLM API dispatch: requests completion (evidence: `eval/evaluator.py`)
- Litellm agent loop → Model providers: resolves provider (evidence: `eval/evaluator.py`)
- Litellm agent loop → MCP client bridge: uses MCP tools (evidence: `eval/evaluator.py`)
- MCP client bridge → MCP server: calls remotely (evidence: `eval/mcp_client.py`)
- MCP server → Session bootstrap: builds tools (evidence: `mcp_app/server.py`)
- Session bootstrap → Kernel tool surface: constructs session (evidence: `mcp_app/session.py`)
- Kernel tool surface → ISA handling: checks ISA (evidence: `mcp_app/agent_tools/base.py`)
- Kernel tool surface → Compile and evaluate: delegates operations (evidence: `mcp_app/agent_tools/base.py`)
- Compile and evaluate → Kernel contracts: uses policy (evidence: `mcp_app/agent_tools/ops.py`)
- Model definitions → Workload generation: generates definitions
- Synthetic problem specs → SIMD loop kernels: benchmarks kernels
- SIMD loop kernels → SIMD loop primitives: uses SIMD macros (evidence: `loops/loop_215.c`)
- Fleet driver → Run logging: logs runs (evidence: `test_scripts/bench_fleet.py`)
- Baseline audits → Compile and evaluate: audits baselines (evidence: `analysis/audit_simd_baselines.py`)
- Fleet driver → Single-shot evaluation: supports single-shot