How thejoshbrod/kernelforge works
Kernel Forge profiles PyTorch operators, generates and validates candidate GPU kernels, then uses MCTS to search for faster implementations while comparing them with the framework baseline. The README also describes remote GPU execution, multi-provider LLM support, live job control, portable snapshots, and deployment-oriented cast packages. The graph covers those product capabilities; some subsystem links are documented in the README or visible in the source index, while unsampled implementations are represented only at supported directory paths.
Main entry points: optimizer/pipeline.py Optimization pipeline · main.py Forge orchestration · benchmarking/pipeline.py Profile workflow · parallel_worker.py Parallel worker
Components 24
Profile and benchmark
- Project and weights
src/projects/artifacts/weights.py - Profile workflow
src/optimizer/benchmarking/pipeline.py - Operator profiling
src/optimizer/benchmarking/profile_project.py - Hardware collectors
src/optimizer/profiling/collectors - Operator benchmarks
src/optimizer/benchmarking/benchmark_ops.py
Kernel generation and search
- Forge orchestration
src/generator/main.py - Operator prompts
src/generator/prompts/prompts.py - Optimization pipeline
src/optimizer/pipeline.py - MCTS search
src/optimizer/core/mcts.py - Kernel generation
src/optimizer/core/generator.py - Parallel worker
src/optimizer/components/worker/parallel_worker.py
Compute backends
- CUDA backend
src/optimizer/backends/cuda - Triton backend
src/optimizer/backends/triton - Remote execution
src/optimizer/core/ssh_client.py - Kernel ABI support
src/optimizer/quantized.py
Runtime services
- LLM providers
src/llm_tools.py - LLM configuration
src/config.py - Usage accounting
src/llm/usage_db.py - Job progress and control
src/progress.py - Optimizer settings
src/optimizer/config/settings.py
Artifacts and deployment
- Anvil snapshots
src/optimizer/tree_store.py - Cast export
paper_benchmarks/paper_bench/cast_export.py - Cast runtime
kernelforge/run_cast.py
Other components
- ML engineer
Connections 25
- ML engineer → Project and weights: creates project
- Project and weights → Profile workflow: starts profiling
- Profile workflow → Operator profiling: runs profiling
src/optimizer/benchmarking/pipeline.py - Profile workflow → Operator benchmarks: runs benchmarks
src/optimizer/benchmarking/pipeline.py - Operator profiling → Hardware collectors: collects hardware data
- ML engineer → Forge orchestration: starts forge
- Forge orchestration → Operator prompts: builds prompts
src/generator/main.py - Forge orchestration → LLM providers: requests generation
src/generator/main.py - Forge orchestration → Optimization pipeline: runs optimization
src/generator/main.py - Optimization pipeline → MCTS search: searches candidates
src/optimizer/pipeline.py - Optimization pipeline → Parallel worker: dispatches work
src/optimizer/pipeline.py - Parallel worker → Kernel generation: generates candidate
src/optimizer/components/worker/parallel_worker.py - Parallel worker → LLM providers: requests optimization
src/optimizer/components/worker/parallel_worker.py - Optimization pipeline → CUDA backend: selects backend
src/optimizer/pipeline.py - Optimization pipeline → Triton backend: selects backend
src/optimizer/pipeline.py - CUDA backend → Remote execution: uses SSH execution
- Triton backend → Remote execution: uses SSH execution
- Forge orchestration → Usage accounting: records usage
src/generator/main.py - LLM providers → LLM configuration: uses provider settings
- ML engineer → Job progress and control: controls job
- ML engineer → Anvil snapshots: saves snapshot
- ML engineer → Cast export: packages model
- Cast export → Cast runtime: creates inference package
- Operator prompts → Kernel ABI support: uses kernel ABI
src/generator/prompts/prompts.py - Optimization pipeline → Optimizer settings: uses settings
src/optimizer/pipeline.py