# tracel-ai/cubecl architecture

> Architecture diagram of the GitHub repository tracel-ai/cubecl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tracel-ai/cubecl
- Repository: https://github.com/tracel-ai/cubecl

## Overview

1. This is a Rust workspace for a JIT-compiled compute language, not a single application. The top-level `cubecl` crate is a thin public entrypoint that re-exports feature-gated pieces of the stack, while most real architecture lives in the sibling crates.

2. The main user-facing flow starts with the `#[cube]` proc macro in `cubecl-macros`. It parses Rust kernel code, rewrites it into ordinary Rust that builds IR at call time, and supports related DSL features such as comptime, autotune annotations, kernel launches, and trait/type generation.

3. `cubecl-core` is the frontend and kernel model. It defines the high-level language constructs for elements, tensors, containers, control flow, runtime options, validation, and post-processing. This crate is where kernels are represented before backend lowering.

4. `cubecl-ir` is the backend-neutral intermediate representation. It models operations such as arithmetic, atomics, branching, barriers, memory, tensor indexing, synchronization, metadata, and runtime properties. Everything downstream consumes this IR, so it is the key architecture boundary between frontend semantics and target codegen.

5. `cubecl-opt` is an IR optimization layer. It runs analysis-driven passes such as liveness, dominance, uniformity, range analysis, and GVN, plus transforms like dead-code elimination, expression merging, disaggregation, and strength reduction. In the architecture diagram, this sits between IR construction and backend code emission.

6. `cubecl-runtime` is the generic execution/runtime layer. It owns compilation caching, autotuning, logging/profiling, stream scheduling, memory management, validation, client/server coordination, and storage abstractions. This crate is the bridge between compiled kernels and actual device execution, independent of any specific backend.

7. Backend-specific runtime crates provide the device-facing implementations:
   - `cubecl-cuda` for NVIDIA/CUDA
   - `cubecl-hip` for AMD/HIP
   - `cubecl-metal` for Apple/Metal
   - `cubecl-wgpu` for WebGPU/Vulkan/Metal-backed execution
   - `cubecl-cpu` for CPU execution and SIMD
   These crates each contain their own device, compute, stream, storage, and server layers, reflecting that scheduling, synchronization, and memory handling differ materially by platform.

8. Code generation is split by target family. `cubecl-cpp` contains shared C++-style codegen helpers for CUDA/HIP/Metal paths, while `cubecl-wgpu` has a WGSL compiler and backend adapters, and `cubecl-spirv` provides a SPIR-V compiler/IR lowering path. The architecture is therefore “one frontend, multiple code generators.”

9. `cubecl-common` is the shared foundation crate for low-level utilities used across the workspace: byte buffers, caching, device handles, profiling, readers, hash/format helpers, quantization, float formats, and assorted infrastructure. It is cross-cutting support rather than a public compute layer.

10. `cubecl-std` is the higher-level standard library on top of the core/runtime stack. It provides reusable tensor/layout/view abstractions, fast math, quantization helpers, swizzle/reinterpret utilities, events, and test coverage for these higher-level operations. This is the library boundary most users would build on after the kernel language itself.

11. `cubecl-zspace` appears to provide shared shape/stride/indexing utilities for tensor-like layouts. It is a supporting domain library for coordinate transforms and layout validation, which feeds into the tensor and view machinery in `cubecl-std` and `cubecl-core`.

12. The execution path is roughly: user Rust kernel -> proc macro expansion -> IR construction in `cubecl-core` -> optimization in `cubecl-opt` / post-processing -> backend-specific lowering -> runtime compilation/cache/autotune -> device dispatch through the selected backend crate. That is the central data flow to diagram.

13. The repository also includes `examples/` and `cubecl-book/` as usage and documentation surfaces, plus `xtask/` for workspace maintenance tasks like build, test, docs, validation, profiling, and book generation. These are important for developer workflow but are not part of the runtime execution architecture.

Main entry points: [`crates/cubecl-runtime/src/lib.rs`](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-runtime/src/lib.rs) (Runtime), [`crates/cubecl-core/src/lib.rs`](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-core/src/lib.rs) (Core frontend), [`crates/cubecl-ir/src/lib.rs`](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-ir/src/lib.rs) (IR), [`crates/cubecl-opt/src/lib.rs`](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-opt/src/lib.rs) (Optimizer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workspace["Workspace"]
  node_cubecl["cubecl<br/>public facade<br/>[lib.rs]"]
  node_examples["Examples<br/>usage samples"]
  node_book["Book<br/>documentation<br/>[overview.md]"]
  node_xtask["xtask<br/>workspace tooling<br/>[main.rs]"]
end

subgraph group_frontend["Frontend"]
  node_cubecl_macros["Macros<br/>proc macro<br/>[lib.rs]"]
  node_cubecl_macros_internal["Macro internals<br/>macro support<br/>[lib.rs]"]
  node_cubecl_core_frontend["Core frontend<br/>kernel frontend<br/>[lib.rs]"]
  node_cubecl_core_codegen["Codegen hooks<br/>frontend codegen<br/>[mod.rs]"]
end

subgraph group_ir["IR"]
  node_cubecl_ir["IR<br/>[lib.rs]"]
  node_cubecl_opt["Optimizer<br/>ir optimization<br/>[lib.rs]"]
end

subgraph group_runtime["Runtime"]
  node_cubecl_runtime["Runtime<br/>[lib.rs]"]
end

subgraph group_backends["Backends"]
  node_cubecl_cpp["C++ codegen<br/>target codegen<br/>[lib.rs]"]
  node_cubecl_spirv["SPIR-V codegen<br/>target compiler<br/>[lib.rs]"]
  node_cubecl_wgpu["WGPU backend<br/>wgsl backend<br/>[lib.rs]"]
  node_cubecl_cpu["CPU backend<br/>cpu runtime<br/>[lib.rs]"]
  node_cubecl_cuda["CUDA backend<br/>cuda runtime<br/>[lib.rs]"]
  node_cubecl_hip["HIP backend<br/>hip runtime<br/>[lib.rs]"]
  node_cubecl_metal["Metal backend<br/>metal runtime<br/>[lib.rs]"]
end

subgraph group_libs["Libraries"]
  node_cubecl_common["Common<br/>shared foundation<br/>[lib.rs]"]
  node_cubecl_std["Std<br/>high-level tensor lib<br/>[lib.rs]"]
  node_cubecl_zspace["Z-space<br/>layout helpers<br/>[lib.rs]"]
end

node_cubecl -->|"re-exports"| node_cubecl_macros
node_cubecl -->|"re-exports"| node_cubecl_core_frontend
node_cubecl -->|"re-exports"| node_cubecl_runtime
node_cubecl_macros -->|"uses"| node_cubecl_macros_internal
node_cubecl_macros -->|"expands into"| node_cubecl_core_frontend
node_cubecl_macros -->|"launches via"| node_cubecl_runtime
node_cubecl_core_frontend -->|"lowers to"| node_cubecl_ir
node_cubecl_core_frontend -->|"integrates"| node_cubecl_core_codegen
node_cubecl_core_frontend -->|"depends on"| node_cubecl_zspace
node_cubecl_core_codegen -->|"emits"| node_cubecl_ir
node_cubecl_ir -->|"optimized by"| node_cubecl_opt
node_cubecl_opt -->|"feeds"| node_cubecl_spirv
node_cubecl_opt -->|"feeds"| node_cubecl_cpp
node_cubecl_opt -->|"feeds"| node_cubecl_wgpu
node_cubecl_opt -->|"feeds"| node_cubecl_cpu
node_cubecl_runtime -->|"depends on"| node_cubecl_common
node_cubecl_runtime -->|"dispatches to"| node_cubecl_cpu
node_cubecl_runtime -->|"dispatches to"| node_cubecl_cuda
node_cubecl_runtime -->|"dispatches to"| node_cubecl_hip
node_cubecl_runtime -->|"dispatches to"| node_cubecl_metal
node_cubecl_runtime -->|"dispatches to"| node_cubecl_wgpu
node_cubecl_std -->|"builds on"| node_cubecl_core_frontend
node_cubecl_std -->|"builds on"| node_cubecl_runtime
node_cubecl_std -->|"uses"| node_cubecl_zspace
node_cubecl_cpp -->|"consumes"| node_cubecl_ir
node_cubecl_wgpu -->|"consumes"| node_cubecl_ir
node_cubecl_spirv -->|"consumes"| node_cubecl_ir
node_cubecl_cpu -->|"uses"| node_cubecl_common
node_cubecl_cuda -->|"uses"| node_cubecl_common
node_cubecl_hip -->|"uses"| node_cubecl_common
node_cubecl_metal -->|"uses"| node_cubecl_common
node_examples -->|"demonstrates"| node_cubecl
node_book -->|"documents"| node_cubecl_std
node_xtask -->|"supports"| node_examples
node_xtask -->|"supports"| node_book

click node_cubecl "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl/src/lib.rs"
click node_cubecl_macros "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-macros/src/lib.rs"
click node_cubecl_macros_internal "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-macros-internal/src/lib.rs"
click node_cubecl_core_frontend "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-core/src/lib.rs"
click node_cubecl_core_codegen "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-core/src/codegen/mod.rs"
click node_cubecl_ir "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-ir/src/lib.rs"
click node_cubecl_opt "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-opt/src/lib.rs"
click node_cubecl_runtime "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-runtime/src/lib.rs"
click node_cubecl_common "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-common/src/lib.rs"
click node_cubecl_std "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-std/src/lib.rs"
click node_cubecl_zspace "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-zspace/src/lib.rs"
click node_cubecl_cpp "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-cpp/src/lib.rs"
click node_cubecl_spirv "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-spirv/src/lib.rs"
click node_cubecl_wgpu "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-wgpu/src/lib.rs"
click node_cubecl_cpu "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-cpu/src/lib.rs"
click node_cubecl_cuda "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-cuda/src/lib.rs"
click node_cubecl_hip "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-hip/src/lib.rs"
click node_cubecl_metal "https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-metal/src/lib.rs"
click node_examples "https://github.com/tracel-ai/cubecl/tree/main/examples"
click node_book "https://github.com/tracel-ai/cubecl/blob/main/cubecl-book/src/overview.md"
click node_xtask "https://github.com/tracel-ai/cubecl/blob/main/xtask/src/main.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cubecl,node_examples,node_book,node_xtask toneBlue
class node_cubecl_macros,node_cubecl_macros_internal,node_cubecl_core_frontend,node_cubecl_core_codegen toneAmber
class node_cubecl_ir,node_cubecl_opt toneMint
class node_cubecl_runtime toneRose
class node_cubecl_cpp,node_cubecl_spirv,node_cubecl_wgpu,node_cubecl_cpu,node_cubecl_cuda,node_cubecl_hip,node_cubecl_metal toneIndigo
class node_cubecl_common,node_cubecl_std,node_cubecl_zspace toneTeal
```

## Components

### Workspace

Top-level entrypoints, docs, examples, and maintenance tooling.

- [cubecl](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl/src/lib.rs): public facade · Thin workspace entrypoint that re-exports feature-gated pieces. · `crates/cubecl/src/lib.rs`
- [Examples](https://github.com/tracel-ai/cubecl/tree/main/examples): usage samples · Workspace examples demonstrating kernels, sharing, fusion, normalization, and tracing. · `examples`
- [Book](https://github.com/tracel-ai/cubecl/blob/main/cubecl-book/src/overview.md): documentation · User and developer documentation for features, usage, and advanced topics. · `cubecl-book/src/overview.md`
- [xtask](https://github.com/tracel-ai/cubecl/blob/main/xtask/src/main.rs): workspace tooling · Maintenance commands for build, test, docs, validation, profiling, and book generation. · `xtask/src/main.rs`

### Frontend

User-facing kernel language, macro expansion, and frontend IR construction.

- [Macros](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-macros/src/lib.rs): proc macro · Parses and rewrites #[cube] kernels, comptime, autotune, launch, and type/trait generation. · `crates/cubecl-macros/src/lib.rs`
- [Macro internals](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-macros-internal/src/lib.rs): macro support · Shared parsing and code-generation helpers for macro expansion. · `crates/cubecl-macros-internal/src/lib.rs`
- [Core frontend](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-core/src/lib.rs): kernel frontend · Kernel language model, element/container abstractions, validation, options, and post-processing. · `crates/cubecl-core/src/lib.rs`
- [Codegen hooks](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-core/src/codegen/mod.rs): frontend codegen · Frontend-side compilation metadata and integration points for lowering kernels. · `crates/cubecl-core/src/codegen/mod.rs`

### IR

Backend-neutral intermediate representation and optimization.

- [IR](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-ir/src/lib.rs): backend-neutral ir · Target-agnostic operations, types, memory, properties, and synchronization model. · `crates/cubecl-ir/src/lib.rs`
- [Optimizer](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-opt/src/lib.rs): ir optimization · Analysis-driven IR passes such as liveness, dominance, uniformity, GVN, DCE, and strength reduction. · `crates/cubecl-opt/src/lib.rs`

### Runtime

Generic execution services shared across backends.

- [Runtime](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-runtime/src/lib.rs): execution runtime · Compilation cache, autotuning, logging, streams, memory management, validation, server/client coordination, and storage. · `crates/cubecl-runtime/src/lib.rs`

### Backends

Backend-specific lowering and device runtime implementations.

- [C++ codegen](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-cpp/src/lib.rs): target codegen · Shared C++-style lowering helpers for CUDA, HIP, and Metal paths. · `crates/cubecl-cpp/src/lib.rs`
- [SPIR-V codegen](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-spirv/src/lib.rs): target compiler · SPIR-V lowering and instruction translation path. · `crates/cubecl-spirv/src/lib.rs`
- [WGPU backend](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-wgpu/src/lib.rs): wgsl backend · WGSL compiler plus backend adapters for WebGPU/Vulkan/Metal-backed execution. · `crates/cubecl-wgpu/src/lib.rs`
- [CPU backend](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-cpu/src/lib.rs): cpu runtime · CPU execution path with compiler, scheduling, streams, and memory handling. · `crates/cubecl-cpu/src/lib.rs`
- [CUDA backend](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-cuda/src/lib.rs): cuda runtime · NVIDIA device runtime with compute, storage, streams, sync, and server layers. · `crates/cubecl-cuda/src/lib.rs`
- [HIP backend](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-hip/src/lib.rs): hip runtime · AMD device runtime with compute, storage, streams, fencing, and server layers. · `crates/cubecl-hip/src/lib.rs`
- [Metal backend](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-metal/src/lib.rs): metal runtime · Apple device runtime with compute, memory, and stream layers. · `crates/cubecl-metal/src/lib.rs`

### Libraries

Shared utility and higher-level tensor support crates.

- [Common](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-common/src/lib.rs): shared foundation · Cross-cutting utilities for buffers, cache, device handles, profiling, hashing, formatting, quantization, and float formats. · `crates/cubecl-common/src/lib.rs`
- [Std](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-std/src/lib.rs): high-level tensor lib · Reusable tensor, layout, view, quantization, fast-math, swizzle, and event abstractions on top of core/runtime. · `crates/cubecl-std/src/lib.rs`
- [Z-space](https://github.com/tracel-ai/cubecl/blob/main/crates/cubecl-zspace/src/lib.rs): layout helpers · Shared shape, stride, indexing, and layout validation utilities for tensor-like coordinates. · `crates/cubecl-zspace/src/lib.rs`

## Connections

- cubecl → Macros: re-exports · Public entrypoint surfaces the proc-macro-driven kernel API.
- cubecl → Core frontend: re-exports · Facade exposes the core frontend and language model.
- cubecl → Runtime: re-exports · Facade exposes runtime services needed by consumers.
- Macros → Macro internals: uses · Macro crate delegates parsing and generation helpers.
- Macros → Core frontend: expands into · Kernel syntax is rewritten into frontend/kernel-building Rust.
- Macros → Runtime: launches via · Expanded kernels call into runtime launch and tuning paths.
- Core frontend → IR: lowers to · Frontend kernel model is represented as backend-neutral IR.
- Core frontend → Codegen hooks: integrates · Frontend carries codegen metadata and compilation integration.
- Core frontend → Z-space: depends on · Tensor/layout semantics rely on shared shape and indexing utilities.
- Codegen hooks → IR: emits · Codegen layer produces or prepares IR-facing compilation artifacts.
- IR → Optimizer: optimized by · IR flows through analysis and transformation passes before code emission.
- Optimizer → SPIR-V codegen: feeds · Optimized IR is lowered toward SPIR-V code generation.
- Optimizer → C++ codegen: feeds · Optimized IR is lowered through shared C++-style backends.
- Optimizer → WGPU backend: feeds · Optimized IR is compiled into WGSL/backend adapters.
- Optimizer → CPU backend: feeds · Optimized IR is compiled into CPU execution code.
- Runtime → Common: depends on · Runtime builds on shared cache, device, profiling, and storage primitives.
- Runtime → CPU backend: dispatches to · Runtime coordinates execution through the selected backend runtime.
- Runtime → CUDA backend: dispatches to · Runtime coordinates execution through the selected backend runtime.
- Runtime → HIP backend: dispatches to · Runtime coordinates execution through the selected backend runtime.
- Runtime → Metal backend: dispatches to · Runtime coordinates execution through the selected backend runtime.
- Runtime → WGPU backend: dispatches to · Runtime coordinates execution through the selected backend runtime.
- Std → Core frontend: builds on · Std library layers tensor/view abstractions over the core language model.
- Std → Runtime: builds on · Std library uses runtime services for execution and memory handling.
- Std → Z-space: uses · Layout/view operations use shared coordinate and stride helpers.
- C++ codegen → IR: consumes · Target-family codegen lowers IR into C++-style dialects.
- WGPU backend → IR: consumes · WGSL compiler and adapters translate IR into web GPU execution artifacts.
- SPIR-V codegen → IR: consumes · SPIR-V compiler lowers backend-neutral IR into SPIR-V instructions.
- CPU backend → Common: uses · CPU runtime shares common buffer, profiling, and device utilities.
- CUDA backend → Common: uses · CUDA runtime shares common buffer, profiling, and device utilities.
- HIP backend → Common: uses · HIP runtime shares common buffer, profiling, and device utilities.
- Metal backend → Common: uses · Metal runtime shares common buffer, profiling, and device utilities.
- Examples → cubecl: demonstrates · Examples show end-user usage of the public facade and kernels.
- Book → Std: documents · Documentation explains higher-level tensor and language features.
- xtask → Examples: supports · Workspace tooling builds and validates examples.
- xtask → Book: supports · Workspace tooling generates and checks the documentation book.