# tile-ai/tilelang architecture

> Architecture diagram of the GitHub repository tile-ai/tilelang, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tile-ai/tilelang
- Repository: https://github.com/tile-ai/tilelang

## Overview

1. TileLang is a Python-first kernel DSL and compiler stack, not a general application framework. The primary user entrypoint is the `tilelang` package, where developers write annotated Python functions that describe tiled GPU/CPU kernels such as GEMM, attention, quantization, and sparse operators.

2. The frontend lives in `tilelang/language` and `tilelang/ir`. It provides the DSL syntax and AST/IR nodes for tensors, loops, allocations, copies, reductions, pipelining, warp/group primitives, and target-specific annotations. This layer is what turns “Pythonic kernel code” into a structured compiler IR.

3. Compilation is centered in `tilelang/engine`, `tilelang/transform`, and `tilelang/backend`. The engine performs semantic checks and lowering orchestration; the transform passes rewrite and legalize the IR; the backend chooses a target-specific codegen/execution path. This is the main compiler pipeline boundary.

4. The C++ core in `src/` implements the lower-level compiler and runtime pieces. `src/op` contains generic operator implementations and helpers, `src/transform` contains IR passes, and `src/runtime` contains logging/error utilities. This code is the performance-critical compiler/runtime substrate beneath the Python API.

5. Backend specialization is explicit. There are separate stacks for `cuda`, `rocm`, `cpu`, `metal`, and `webgpu`, each with its own codegen, target utilities, runtime hooks, and operator implementations. That means the architecture is multi-backend by design rather than “CUDA-only with portability shims.”

6. NVIDIA-specific lowering is the most extensive backend. `src/cuda` and `tilelang/cuda` include CUDA codegen, PTX/intrinsic rules, runtime stubs, warp-group/TMA/WGMMA handling, and transforms for Hopper/Blackwell-era features. There is also a `tilelang/contrib/cutedsl` path and `src/cuda/codegen/codegen_cutedsl.cc`, showing an alternate code generation route to CUTLASS CuTe DSL.

7. AMD support is a parallel architecture, not an afterthought. `src/rocm`, `tilelang/rocm`, and `3rdparty/hip-headers` provide HIP/ROCm codegen and MFMA/WMMA intrinsic support. The repo also ships AMD-specific examples and tests, so AMD compilation, lowering, and runtime behavior are first-class.

8. The Python runtime and JIT layer sits between user kernels and compiled shared libraries. `tilelang/jit` contains adapters for NVRTC, Cython, CuTeDSL, Torch, and TVM FFI, plus kernel caching and wrapper generation. This layer is responsible for compiling on demand, loading the result, and presenting callable kernels back to Python.

9. Execution is cached and target-aware. `tilelang/cache` and the JIT adapter cache modules store compiled artifacts, while `tilelang/backend/execution_backend.py` and target modules manage how kernels are launched. The data flow is: user kernel → IR/lowers → backend codegen → compiled artifact → cached loader/execution.

10. Tile-specific performance modeling and operator synthesis live in `tilelang/carver` and `tilelang/tileop`. `carver` analyzes shapes/architectures and suggests schedules or hints; `tileop` holds reusable operator templates/registries such as GEMM and sparse GEMM. These subsystems support autotuning and template-driven specialization.

11. Higher-level algorithm support is organized into reusable operator families in `tilelang/cuda/op`, `tilelang/rocm/op`, `tilelang/cpu/op`, `tilelang/metal/op`, and the corresponding Python wrappers. This separates “what the operator does” from “how it is lowered for each device class,” which is important for diagramming the architecture cleanly.

12. The repository includes a substantial verification and regression layer. `testing/` contains compiler-pass tests, backend tests, runtime tests, JIT tests, and issue regressions; `examples/` contains end-to-end operator demonstrations and benchmarks; `benchmark/` holds reproducible performance scripts. These directories form the quality and performance validation boundary around the compiler.

13. Build and distribution are managed as a mixed Python/C++ project. `CMakeLists.txt`, the `cmake/` helpers, `pyproject.toml`, and the CI/docker files coordinate native compilation, Python packaging, wheel builds, docs, and backend-specific environment setup. The vendored `3rdparty/tvm`, CUTLASS, composable kernel, and HIP headers are foundational dependencies that materially shape the compiler and backend architecture.

Main entry points: [`tilelang/backend`](https://github.com/tile-ai/tilelang/tree/main/tilelang/backend) (Backend API), [`src/cuda`](https://github.com/tile-ai/tilelang/tree/main/src/cuda) (CUDA Core), [`src/rocm`](https://github.com/tile-ai/tilelang/tree/main/src/rocm) (ROCm Core), [`CMakeLists.txt`](https://github.com/tile-ai/tilelang/blob/main/CMakeLists.txt) (Build System)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_frontend["Frontend DSL"]
  node_n_tilelang_pkg["tilelang<br/>Python entrypoint<br/>[__init__.py]"]
  node_n_language["Language DSL<br/>DSL frontend"]
  node_n_ir_layout["IR & Layout<br/>IR model<br/>[ir.py]"]
  node_n_tileop["TileOps<br/>operator templates"]
  node_n_carver["Carver<br/>scheduling hints"]
end

subgraph group_g_compile["Compiler Pipeline"]
  node_n_engine["Engine<br/>Lowering orchestrator"]
  node_n_transforms_py["Python Passes<br/>rewrite passes"]
  node_n_backend_py["Backend API<br/>target dispatch"]
end

subgraph group_g_backends["Backends"]
  node_n_cuda_py["CUDA Stack<br/>CUDA backend"]
  node_n_rocm_py["ROCm Stack<br/>ROCm backend"]
  node_n_cpu_py["CPU Stack<br/>CPU backend"]
  node_n_metal_py["Metal Stack<br/>Metal backend"]
  node_n_webgpu_py["WebGPU Stack<br/>WebGPU backend"]
end

subgraph group_g_runtime["JIT and Runtime"]
  node_n_jit["JIT Layer<br/>on-demand build"]
  node_n_cache[("Kernel Cache<br/>artifact cache")]
end

subgraph group_g_native["Native Core"]
  node_n_src_op["Core Ops<br/>C++ ops"]
  node_n_src_transform["Core Transforms<br/>C++ passes"]
  node_n_src_cuda["CUDA Core<br/>C++ CUDA"]
  node_n_src_rocm["ROCm Core<br/>C++ HIP"]
  node_n_src_runtime["Runtime Core<br/>C++ runtime"]
end

subgraph group_g_quality["Validation"]
  node_n_examples_tests["Examples & Tests<br/>validation"]
  node_n_build["Build System<br/>build/distribution<br/>[CMakeLists.txt]"]
  node_n_deps[("Vendored Deps<br/>third-party")]
end

node_n_tilelang_pkg -->|"uses DSL"| node_n_language
node_n_language -->|"builds IR"| node_n_ir_layout
node_n_ir_layout -->|"lowered by"| node_n_engine
node_n_engine -->|"applies passes"| node_n_transforms_py
node_n_transforms_py -->|"targets backends"| node_n_backend_py
node_n_backend_py -->|"executes via"| node_n_jit
node_n_jit -->|"stores artifacts"| node_n_cache
node_n_cache -->|"reuses target code"| node_n_backend_py
node_n_backend_py -->|"dispatches"| node_n_cuda_py
node_n_backend_py -->|"dispatches"| node_n_rocm_py
node_n_backend_py -->|"dispatches"| node_n_cpu_py
node_n_backend_py -->|"dispatches"| node_n_metal_py
node_n_backend_py -->|"dispatches"| node_n_webgpu_py
node_n_tileop -->|"guided by"| node_n_carver
node_n_tileop -->|"specialized by"| node_n_engine
node_n_src_op -->|"consumed by"| node_n_src_transform
node_n_src_transform -->|"specialized in"| node_n_src_cuda
node_n_src_transform -->|"specialized in"| node_n_src_rocm
node_n_src_cuda -->|"implements"| node_n_src_op
node_n_src_rocm -->|"implements"| node_n_src_op
node_n_src_cuda -->|"uses runtime"| node_n_src_runtime
node_n_src_rocm -->|"uses runtime"| node_n_src_runtime
node_n_examples_tests -->|"exercises"| node_n_tilelang_pkg
node_n_build -->|"pulls in"| node_n_deps
node_n_build -->|"builds"| node_n_src_cuda
node_n_build -->|"builds"| node_n_src_rocm

click node_n_tilelang_pkg "https://github.com/tile-ai/tilelang/blob/main/tilelang/__init__.py"
click node_n_language "https://github.com/tile-ai/tilelang/tree/main/tilelang/language"
click node_n_ir_layout "https://github.com/tile-ai/tilelang/blob/main/tilelang/ir.py"
click node_n_engine "https://github.com/tile-ai/tilelang/tree/main/tilelang/engine"
click node_n_transforms_py "https://github.com/tile-ai/tilelang/tree/main/tilelang/transform"
click node_n_backend_py "https://github.com/tile-ai/tilelang/tree/main/tilelang/backend"
click node_n_jit "https://github.com/tile-ai/tilelang/tree/main/tilelang/jit"
click node_n_cache "https://github.com/tile-ai/tilelang/tree/main/tilelang/cache"
click node_n_cuda_py "https://github.com/tile-ai/tilelang/tree/main/tilelang/cuda"
click node_n_rocm_py "https://github.com/tile-ai/tilelang/tree/main/tilelang/rocm"
click node_n_cpu_py "https://github.com/tile-ai/tilelang/tree/main/tilelang/cpu"
click node_n_metal_py "https://github.com/tile-ai/tilelang/tree/main/tilelang/metal"
click node_n_webgpu_py "https://github.com/tile-ai/tilelang/tree/main/tilelang/webgpu"
click node_n_tileop "https://github.com/tile-ai/tilelang/tree/main/tilelang/tileop"
click node_n_carver "https://github.com/tile-ai/tilelang/tree/main/tilelang/carver"
click node_n_src_op "https://github.com/tile-ai/tilelang/tree/main/src/op"
click node_n_src_transform "https://github.com/tile-ai/tilelang/tree/main/src/transform"
click node_n_src_cuda "https://github.com/tile-ai/tilelang/tree/main/src/cuda"
click node_n_src_rocm "https://github.com/tile-ai/tilelang/tree/main/src/rocm"
click node_n_src_runtime "https://github.com/tile-ai/tilelang/tree/main/src/runtime"
click node_n_examples_tests "https://github.com/tile-ai/tilelang/tree/main/examples"
click node_n_build "https://github.com/tile-ai/tilelang/blob/main/CMakeLists.txt"
click node_n_deps "https://github.com/tile-ai/tilelang/tree/main/3rdparty"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_tilelang_pkg,node_n_language,node_n_ir_layout,node_n_tileop,node_n_carver toneBlue
class node_n_engine,node_n_transforms_py,node_n_backend_py toneAmber
class node_n_cuda_py,node_n_rocm_py,node_n_cpu_py,node_n_metal_py,node_n_webgpu_py toneMint
class node_n_jit,node_n_cache toneRose
class node_n_src_op,node_n_src_transform,node_n_src_cuda,node_n_src_rocm,node_n_src_runtime toneIndigo
class node_n_examples_tests,node_n_build,node_n_deps toneTeal
```

## Components

### Frontend DSL

Python-facing kernel language, AST/IR, and layout abstractions.

- [tilelang](https://github.com/tile-ai/tilelang/blob/main/tilelang/__init__.py): Python entrypoint · Top-level Python API for authoring tiled kernels and invoking compilation/execution. · `tilelang/__init__.py`
- [Language DSL](https://github.com/tile-ai/tilelang/tree/main/tilelang/language): DSL frontend · Annotated kernel syntax, parser, AST helpers, loop/copy/reduce primitives, and eager/TIR entrypoints. · `tilelang/language`
- [IR & Layout](https://github.com/tile-ai/tilelang/blob/main/tilelang/ir.py): IR model · Structured kernel IR plus tensor/layout abstractions used by the compiler and backends. · `tilelang/ir.py`
- [TileOps](https://github.com/tile-ai/tilelang/tree/main/tilelang/tileop): operator templates · Reusable operator registries and templates such as GEMM and sparse GEMM. · `tilelang/tileop`
- [Carver](https://github.com/tile-ai/tilelang/tree/main/tilelang/carver): scheduling hints · Performance analysis, shape/arch reasoning, and schedule hint generation for autotuning. · `tilelang/carver`

### Compiler Pipeline

Semantic checks and lowering/rewriting passes that turn DSL kernels into target IR.

- [Engine](https://github.com/tile-ai/tilelang/tree/main/tilelang/engine): Lowering orchestrator · Semantic checks, lowering entrypoints, and callback hooks for compilation. · `tilelang/engine`
- [Python Passes](https://github.com/tile-ai/tilelang/tree/main/tilelang/transform): rewrite passes · High-level IR transforms and legality rewrites before backend specialization. · `tilelang/transform`
- [Backend API](https://github.com/tile-ai/tilelang/tree/main/tilelang/backend): target dispatch · Target selection, device codegen glue, and execution backend interfaces. · `tilelang/backend`

### Backends

Target-specific codegen, execution, intrinsics, and operator families.

- [CUDA Stack](https://github.com/tile-ai/tilelang/tree/main/tilelang/cuda): CUDA backend · Python CUDA codegen, execution backend, intrinsics, and CUDA operators. · `tilelang/cuda`
- [ROCm Stack](https://github.com/tile-ai/tilelang/tree/main/tilelang/rocm): ROCm backend · Python ROCm codegen, execution backend, intrinsics, and HIP operators. · `tilelang/rocm`
- [CPU Stack](https://github.com/tile-ai/tilelang/tree/main/tilelang/cpu): CPU backend · Python CPU codegen, execution backend, and CPU operator implementations. · `tilelang/cpu`
- [Metal Stack](https://github.com/tile-ai/tilelang/tree/main/tilelang/metal): Metal backend · Python Metal codegen, execution backend, and Metal operator implementations. · `tilelang/metal`
- [WebGPU Stack](https://github.com/tile-ai/tilelang/tree/main/tilelang/webgpu): WebGPU backend · Python WebGPU codegen and pipeline hooks. · `tilelang/webgpu`

### JIT and Runtime

On-demand compilation, caching, loading, and execution wrappers.

- [JIT Layer](https://github.com/tile-ai/tilelang/tree/main/tilelang/jit): on-demand build · Adapters for NVRTC, Cython, CuTeDSL, Torch, and TVM FFI plus wrapper generation and kernel loading. · `tilelang/jit`
- [Kernel Cache](https://github.com/tile-ai/tilelang/tree/main/tilelang/cache): artifact cache · Stores compiled artifacts and compiled-kernel metadata for reuse. · `tilelang/cache`

### Native Core

C++ compiler/runtime substrate and backend-specific lowering implementations.

- [Core Ops](https://github.com/tile-ai/tilelang/tree/main/src/op): C++ ops · Shared operator implementations and helpers used across targets. · `src/op`
- [Core Transforms](https://github.com/tile-ai/tilelang/tree/main/src/transform): C++ passes · Lower-level IR rewrites, legalization, pipeline planning, and layout inference in C++. · `src/transform`
- [CUDA Core](https://github.com/tile-ai/tilelang/tree/main/src/cuda): C++ CUDA · CUDA codegen, intrinsics, transforms, runtime stubs, and CUDA operator lowerings. · `src/cuda`
- [ROCm Core](https://github.com/tile-ai/tilelang/tree/main/src/rocm): C++ HIP · ROCm codegen, intrinsics, transforms, runtime stubs, and HIP operator lowerings. · `src/rocm`
- [Runtime Core](https://github.com/tile-ai/tilelang/tree/main/src/runtime): C++ runtime · Logging, error helpers, and support utilities for generated code/runtime execution. · `src/runtime`

### Validation

Tests, benchmarks, and examples that exercise the stack end to end.

- [Examples & Tests](https://github.com/tile-ai/tilelang/tree/main/examples): validation · End-to-end examples, regressions, backend tests, and performance benchmarks. · `examples`
- [Build System](https://github.com/tile-ai/tilelang/blob/main/CMakeLists.txt): build/distribution · CMake, Python packaging, CI, and container setup for mixed Python/C++ builds. · `CMakeLists.txt`
- [Vendored Deps](https://github.com/tile-ai/tilelang/tree/main/3rdparty): third-party · Embedded dependencies shaping compiler and backend behavior, including TVM, CUTLASS, composable kernel, and HIP headers. · `3rdparty`

## Connections

- tilelang → Language DSL: uses DSL · Users author kernels through the Python language layer.
- Language DSL → IR & Layout: builds IR · The frontend turns Pythonic kernel code into structured IR and layouts.
- IR & Layout → Engine: lowered by · IR is handed to engine semantic checks and lowering.
- Engine → Python Passes: applies passes · The engine orchestrates high-level rewriting and legalization passes.
- Python Passes → Backend API: targets backends · Transforms prepare the IR for backend-specific codegen and execution.
- Backend API → JIT Layer: executes via · Backends feed compiled artifacts into the JIT/load path.
- JIT Layer → Kernel Cache: stores artifacts · Compiled kernels are cached for reuse across invocations.
- Kernel Cache → Backend API: reuses target code · Cached artifacts remain target-aware and are loaded through backend execution.
- Backend API → CUDA Stack: dispatches · CUDA is a primary specialized backend path.
- Backend API → ROCm Stack: dispatches · ROCm is a first-class specialized backend path.
- Backend API → CPU Stack: dispatches · CPU codegen and execution are supported as a separate backend.
- Backend API → Metal Stack: dispatches · Metal support is integrated through its own backend stack.
- Backend API → WebGPU Stack: dispatches · WebGPU support follows its own codegen path.
- TileOps → Carver: guided by · Operator templates can be analyzed and tuned with schedule hints.
- TileOps → Engine: specialized by · TileOps feed reusable operator structure into compilation.
- Core Ops → Core Transforms: consumed by · Core operator semantics are rewritten by native transform passes.
- Core Transforms → CUDA Core: specialized in · CUDA-specific lowering adds async copy, barriers, tensor cores, and warp-group features.
- Core Transforms → ROCm Core: specialized in · ROCm-specific lowering adds HIP/MFMA/WMMA handling.
- CUDA Core → Core Ops: implements · CUDA operator lowerings back shared op abstractions.
- ROCm Core → Core Ops: implements · ROCm operator lowerings back shared op abstractions.
- CUDA Core → Runtime Core: uses runtime · CUDA runtime hooks rely on common runtime support.
- ROCm Core → Runtime Core: uses runtime · ROCm runtime hooks rely on common runtime support.
- Examples & Tests → tilelang: exercises · Examples and tests validate the public Python API and end-to-end compilation.
- Build System → Vendored Deps: pulls in · Build and packaging are shaped by vendored native dependencies.
- Build System → CUDA Core: builds · Native build configuration compiles backend-specific C++ code.
- Build System → ROCm Core: builds · Native build configuration compiles ROCm-specific C++ code.
