How xjn-la-la-land/amx-gemm-handwritten works
This repository implements and benchmarks int8 AMX GEMM in offline-prepacked and online-packing variants. The main workflow is to configure matrix sizes and benchmark conditions, initialize a variant-specific plan, run repeated AMX computations, and record performance results for analysis. The README also documents CPU/perf-based experiments and plotting utilities. Sampled sources show harness planning and compute paths, but not every variant’s entry-point wiring; the graph omits uncertain links.
Main entry points: bench_harness.hpp Benchmark harness · online-packing/gemm-test.cpp Online variant · offline-packing/gemm-test.cpp Offline variant · bench.sh Benchmark orchestration
Components 18
Benchmark runtime
- CLI and config
src/common/bench_harness.hpp - Benchmark harness
src/common/bench_harness.hpp - Matrix inputs
src/common/bench_harness.hpp - NUMA threading
src/common/thread_params.hpp - CSV results
src/common/bench_harness.hpp
GEMM variants
- Offline variant
src/offline-packing/gemm-test.cpp - Offline packing
src/offline-packing/amx-gemm.cpp - Online variant
src/online-packing/gemm-test.cpp - Online packing
src/online-packing/amx-gemm.cpp - Block autotuning
src/online-packing/autotune.hpp - Packed buffers
src/common/page_alloc.hpp
Experiment controls
- Benchmark orchestration
scripts/bench.sh - CPU controls
scripts/cpu-isolation.py - Frequency control
scripts/freq.sh - Perf counters
Results analysis
- Plot utilities
tools/plot_amx_util.py - Stage plots
tools/plot_stage_breakdown.py
Other components
- Engineer
Connections 19
- Engineer → CLI and config: sets options
- Engineer → Benchmark orchestration: runs benchmark
- CLI and config → Benchmark harness: configures
- Benchmark harness → Matrix inputs: creates matrices
- Benchmark harness → Offline variant: selects variant
- Benchmark harness → Online variant: selects variant
- Offline variant → Offline packing: pre-packs
- Offline variant → Packed buffers: allocates packed data
- Offline variant → CSV results: records metrics
- Online variant → Online packing: computes with packing
- Online variant → Block autotuning: tunes blocks
- Online variant → Packed buffers: allocates packed data
- Online variant → CSV results: records metrics
- Benchmark harness → NUMA threading: uses thread settings
- Benchmark orchestration → Frequency control: locks frequency
- Benchmark orchestration → CPU controls: isolates CPUs
- Benchmark orchestration → Perf counters: invokes counters
- CSV results → Plot utilities: feeds utilization plots
- CSV results → Stage plots: feeds stage plots