# thejoshbrod/kernelforge architecture

> Architecture diagram of the GitHub repository thejoshbrod/kernelforge, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thejoshbrod/kernelforge
- Repository: https://github.com/thejoshbrod/kernelforge

## Overview

Kernel Forge profiles PyTorch operators, generates and validates candidate GPU kernels, then uses MCTS to search for faster implementations while comparing them with the framework baseline. The README also describes remote GPU execution, multi-provider LLM support, live job control, portable snapshots, and deployment-oriented cast packages. The graph covers those product capabilities; some subsystem links are documented in the README or visible in the source index, while unsampled implementations are represented only at supported directory paths.

Main entry points: [`src/optimizer/pipeline.py`](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/pipeline.py) (Optimization pipeline), [`src/generator/main.py`](https://github.com/thejoshbrod/kernelforge/blob/main/src/generator/main.py) (Forge orchestration), [`src/optimizer/benchmarking/pipeline.py`](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/benchmarking/pipeline.py) (Profile workflow), [`src/optimizer/components/worker/parallel_worker.py`](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/components/worker/parallel_worker.py) (Parallel worker)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_profile["Profile and benchmark"]
  node_project["Project and weights<br/>[weights.py]"]
  node_benchmark_flow["Profile workflow<br/>[pipeline.py]"]
  node_profile_ops["Operator profiling<br/>[profile_project.py]"]
  node_profile_collect["Hardware collectors"]
  node_benchmark_ops["Operator benchmarks<br/>[benchmark_ops.py]"]
end

subgraph group_search["Kernel generation and search"]
  node_forge["Forge orchestration<br/>[main.py]"]
  node_operator_prompts["Operator prompts<br/>[prompts.py]"]
  node_optimizer["Optimization pipeline<br/>[pipeline.py]"]
  node_mcts["MCTS search<br/>[mcts.py]"]
  node_kernel_generator["Kernel generation<br/>[generator.py]"]
  node_parallel_worker["Parallel worker<br/>[parallel_worker.py]"]
end

subgraph group_compute["Compute backends"]
  node_cuda["CUDA backend"]
  node_triton["Triton backend"]
  node_remote["Remote execution<br/>[ssh_client.py]"]
  node_quantized["Kernel ABI support<br/>[quantized.py]"]
end

subgraph group_services["Runtime services"]
  node_llm_api["LLM providers<br/>[llm_tools.py]"]
  node_llm_config["LLM configuration<br/>[config.py]"]
  node_usage[("Usage accounting<br/>[usage_db.py]")]
  node_job_state["Job progress and control<br/>[progress.py]"]
  node_runtime_settings["Optimizer settings<br/>[settings.py]"]
end

subgraph group_artifacts["Artifacts and deployment"]
  node_snapshots["Anvil snapshots<br/>[tree_store.py]"]
  node_cast_export["Cast export<br/>[cast_export.py]"]
  node_cast_runtime["Cast runtime<br/>[run_cast.py]"]
end

node_engineer(("ML engineer"))

node_engineer -->|"creates project"| node_project
node_project -->|"starts profiling"| node_benchmark_flow
node_benchmark_flow -->|"runs profiling"| node_profile_ops
node_benchmark_flow -->|"runs benchmarks"| node_benchmark_ops
node_profile_ops -.->|"collects hardware data"| node_profile_collect
node_engineer -->|"starts forge"| node_forge
node_forge -->|"builds prompts"| node_operator_prompts
node_forge -->|"requests generation"| node_llm_api
node_forge -->|"runs optimization"| node_optimizer
node_optimizer -->|"searches candidates"| node_mcts
node_optimizer -->|"dispatches work"| node_parallel_worker
node_parallel_worker -->|"generates candidate"| node_kernel_generator
node_parallel_worker -->|"requests optimization"| node_llm_api
node_optimizer -->|"selects backend"| node_cuda
node_optimizer -->|"selects backend"| node_triton
node_cuda -.->|"uses SSH execution"| node_remote
node_triton -.->|"uses SSH execution"| node_remote
node_forge -->|"records usage"| node_usage
node_llm_api -.->|"uses provider settings"| node_llm_config
node_engineer -->|"controls job"| node_job_state
node_engineer -.->|"saves snapshot"| node_snapshots
node_engineer -->|"packages model"| node_cast_export
node_cast_export -.->|"creates inference package"| node_cast_runtime
node_operator_prompts -->|"uses kernel ABI"| node_quantized
node_optimizer -->|"uses settings"| node_runtime_settings

click node_project "https://github.com/thejoshbrod/kernelforge/blob/main/src/projects/artifacts/weights.py"
click node_benchmark_flow "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/benchmarking/pipeline.py"
click node_profile_ops "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/benchmarking/profile_project.py"
click node_profile_collect "https://github.com/thejoshbrod/kernelforge/tree/main/src/optimizer/profiling/collectors"
click node_benchmark_ops "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/benchmarking/benchmark_ops.py"
click node_forge "https://github.com/thejoshbrod/kernelforge/blob/main/src/generator/main.py"
click node_operator_prompts "https://github.com/thejoshbrod/kernelforge/blob/main/src/generator/prompts/prompts.py"
click node_optimizer "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/pipeline.py"
click node_mcts "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/core/mcts.py"
click node_kernel_generator "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/core/generator.py"
click node_parallel_worker "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/components/worker/parallel_worker.py"
click node_llm_api "https://github.com/thejoshbrod/kernelforge/blob/main/src/llm_tools.py"
click node_llm_config "https://github.com/thejoshbrod/kernelforge/blob/main/src/config.py"
click node_usage "https://github.com/thejoshbrod/kernelforge/blob/main/src/llm/usage_db.py"
click node_job_state "https://github.com/thejoshbrod/kernelforge/blob/main/src/progress.py"
click node_runtime_settings "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/config/settings.py"
click node_cuda "https://github.com/thejoshbrod/kernelforge/tree/main/src/optimizer/backends/cuda"
click node_triton "https://github.com/thejoshbrod/kernelforge/tree/main/src/optimizer/backends/triton"
click node_remote "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/core/ssh_client.py"
click node_quantized "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/quantized.py"
click node_snapshots "https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/tree_store.py"
click node_cast_export "https://github.com/thejoshbrod/kernelforge/blob/main/paper_benchmarks/paper_bench/cast_export.py"
click node_cast_runtime "https://github.com/thejoshbrod/kernelforge/blob/main/kernelforge/run_cast.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_project,node_benchmark_flow,node_profile_ops,node_profile_collect,node_benchmark_ops toneBlue
class node_forge,node_operator_prompts,node_optimizer,node_mcts,node_kernel_generator,node_parallel_worker toneAmber
class node_cuda,node_triton,node_remote,node_quantized toneMint
class node_llm_api,node_llm_config,node_usage,node_job_state,node_runtime_settings toneRose
class node_snapshots,node_cast_export,node_cast_runtime,node_engineer toneIndigo
```

## Components

### Profile and benchmark

- [Project and weights](https://github.com/thejoshbrod/kernelforge/blob/main/src/projects/artifacts/weights.py): `src/projects/artifacts/weights.py`
- [Profile workflow](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/benchmarking/pipeline.py): `src/optimizer/benchmarking/pipeline.py`
- [Operator profiling](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/benchmarking/profile_project.py): `src/optimizer/benchmarking/profile_project.py`
- [Hardware collectors](https://github.com/thejoshbrod/kernelforge/tree/main/src/optimizer/profiling/collectors): `src/optimizer/profiling/collectors`
- [Operator benchmarks](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/benchmarking/benchmark_ops.py): `src/optimizer/benchmarking/benchmark_ops.py`

### Kernel generation and search

- [Forge orchestration](https://github.com/thejoshbrod/kernelforge/blob/main/src/generator/main.py): `src/generator/main.py`
- [Operator prompts](https://github.com/thejoshbrod/kernelforge/blob/main/src/generator/prompts/prompts.py): `src/generator/prompts/prompts.py`
- [Optimization pipeline](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/pipeline.py): `src/optimizer/pipeline.py`
- [MCTS search](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/core/mcts.py): `src/optimizer/core/mcts.py`
- [Kernel generation](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/core/generator.py): `src/optimizer/core/generator.py`
- [Parallel worker](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/components/worker/parallel_worker.py): `src/optimizer/components/worker/parallel_worker.py`

### Compute backends

- [CUDA backend](https://github.com/thejoshbrod/kernelforge/tree/main/src/optimizer/backends/cuda): `src/optimizer/backends/cuda`
- [Triton backend](https://github.com/thejoshbrod/kernelforge/tree/main/src/optimizer/backends/triton): `src/optimizer/backends/triton`
- [Remote execution](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/core/ssh_client.py): `src/optimizer/core/ssh_client.py`
- [Kernel ABI support](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/quantized.py): `src/optimizer/quantized.py`

### Runtime services

- [LLM providers](https://github.com/thejoshbrod/kernelforge/blob/main/src/llm_tools.py): `src/llm_tools.py`
- [LLM configuration](https://github.com/thejoshbrod/kernelforge/blob/main/src/config.py): `src/config.py`
- [Usage accounting](https://github.com/thejoshbrod/kernelforge/blob/main/src/llm/usage_db.py): `src/llm/usage_db.py`
- [Job progress and control](https://github.com/thejoshbrod/kernelforge/blob/main/src/progress.py): `src/progress.py`
- [Optimizer settings](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/config/settings.py): `src/optimizer/config/settings.py`

### Artifacts and deployment

- [Anvil snapshots](https://github.com/thejoshbrod/kernelforge/blob/main/src/optimizer/tree_store.py): `src/optimizer/tree_store.py`
- [Cast export](https://github.com/thejoshbrod/kernelforge/blob/main/paper_benchmarks/paper_bench/cast_export.py): `paper_benchmarks/paper_bench/cast_export.py`
- [Cast runtime](https://github.com/thejoshbrod/kernelforge/blob/main/kernelforge/run_cast.py): `kernelforge/run_cast.py`

### Other components

- **ML engineer**

## Connections

- ML engineer → Project and weights: creates project
- Project and weights → Profile workflow: starts profiling
- Profile workflow → Operator profiling: runs profiling (evidence: `src/optimizer/benchmarking/pipeline.py`)
- Profile workflow → Operator benchmarks: runs benchmarks (evidence: `src/optimizer/benchmarking/pipeline.py`)
- Operator profiling → Hardware collectors: collects hardware data
- ML engineer → Forge orchestration: starts forge
- Forge orchestration → Operator prompts: builds prompts (evidence: `src/generator/main.py`)
- Forge orchestration → LLM providers: requests generation (evidence: `src/generator/main.py`)
- Forge orchestration → Optimization pipeline: runs optimization (evidence: `src/generator/main.py`)
- Optimization pipeline → MCTS search: searches candidates (evidence: `src/optimizer/pipeline.py`)
- Optimization pipeline → Parallel worker: dispatches work (evidence: `src/optimizer/pipeline.py`)
- Parallel worker → Kernel generation: generates candidate (evidence: `src/optimizer/components/worker/parallel_worker.py`)
- Parallel worker → LLM providers: requests optimization (evidence: `src/optimizer/components/worker/parallel_worker.py`)
- Optimization pipeline → CUDA backend: selects backend (evidence: `src/optimizer/pipeline.py`)
- Optimization pipeline → Triton backend: selects backend (evidence: `src/optimizer/pipeline.py`)
- CUDA backend → Remote execution: uses SSH execution
- Triton backend → Remote execution: uses SSH execution
- Forge orchestration → Usage accounting: records usage (evidence: `src/generator/main.py`)
- LLM providers → LLM configuration: uses provider settings
- ML engineer → Job progress and control: controls job
- ML engineer → Anvil snapshots: saves snapshot
- ML engineer → Cast export: packages model
- Cast export → Cast runtime: creates inference package
- Operator prompts → Kernel ABI support: uses kernel ABI (evidence: `src/generator/prompts/prompts.py`)
- Optimization pipeline → Optimizer settings: uses settings (evidence: `src/optimizer/pipeline.py`)
