# microsoft/tilefusion architecture

> Architecture diagram of the GitHub repository microsoft/tilefusion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/tilefusion
- Repository: https://github.com/microsoft/tilefusion

## Overview

##### Project type and purpose
- TileFusion is a high-level, modular C++/CUDA library for GPU tile processing with optional Python bindings (PyTorch extension).
- It provides header-only C++ primitives for tile layout, iterators, loaders/storers, and exposes select kernels (e.g., flash attention, fused GEMMs) to Python.

##### File structure
- Top-level build and config:
- CMakeLists.txt, Makefile, cmake/ (dependencies, external), .github/workflows, scripts/
- setup.py, pyproject.toml, requirements.txt (Python packaging & dev)
- Core C++ library:
- / (public headers: cell, kernels, types, util, traits)
- src/ (kernel implementations .cu, torch_bind.cc, cuda_utils.cc)
- Third-party dependencies:
- 3rd-party/cutlass, 3rd-party/googletest
- Examples & benchmarks:
- examples/ (CUDA code snippets showing usage patterns)
- benchmarks/ (C++ and Python scripts measuring performance)
- Tests:
- tests/cpp (unit tests via GoogleTest and CUDA kernels)
- tests/python (pytest suites for Python ops)

##### README for additional insights
- Core concepts: separation of data flow vs. kernel configuration; three-level GPU memory abstraction (global, shared, register).
- Usage modes: header-only C++ library or Python package (via setup.py).
- Dependencies: C++20, CUDA 12+, PyTorch for Python.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Build & Test Infrastructure
    subgraph "Build System" 
        direction TB
        BM[CMakeLists.txt]:::build
        MK[Makefile]:::build
        CM[cmake/]:::build
        SP[setup.py]:::build
        PP[pyproject.toml]:::build
    end

    %% Core C++ Library
    subgraph "Core C++ Library" 
        direction TB
        subgraph "Public Headers"
            direction TB
            IH[include/]:::cpp
            CH[cell/]:::cpp
            KH[kernels/]:::cpp
            TH[types/]:::cpp
            UH[util/]:::cpp
            TR[traits/]:::cpp
        end
        subgraph "CUDA & Utilities"
            direction TB
            SK[src/kernels/]:::cuda
            CU[cuda_utils.cc]:::cpp
            CI[cuda_info.cc]:::cpp
            CUH[cuda_utils.hpp]:::cpp
            CIH[cuda_info.hpp]:::cpp
            KR[kernel_registry.hpp]:::cpp
        end
    end

    %% Python Extension
    subgraph "Python Extension" 
        direction TB
        TBIND[src/torch_bind.cc]:::cpp
        subgraph "Python API"
            direction TB
            PYI[python/__init__.py]:::py
            PYV[python/__version__.py]:::py
            FA[python/ops/flash_attention.py]:::py
            SN[python/ops/scatter_nd.py]:::py
        end
    end

    %% Examples & Benchmarks
    subgraph "Examples & Benchmarks"
        direction TB
        subgraph Examples
            E1[examples/01_gemm/]:::cpp
            E2[examples/02_fused_two_gemms/]:::cpp
            E3[examples/03_flash_attention/]:::cpp
        end
        subgraph Benchmarks
            direction TB
            BC[benchmarks/cpp/]:::cpp
            BP[benchmarks/utils/python/]:::py
        end
    end

    %% Tests
    subgraph "Tests"
        direction TB
        CPPT[tests/cpp/]:::cpp
        PYT[tests/python/]:::py
    end

    %% External Dependencies
    subgraph "External Dependencies"
        direction TB
        CUT[3rd-party/cutlass/]:::ext
        GTEST[3rd-party/googletest/]:::ext
        CUDA[CUDA Toolkit]:::ext
        TORCH[PyTorch]:::ext
    end

    %% Build Relationships
    BM -->|"configures"| CM
    BM -->|compiles| IH
    BM -->|compiles| SK
    BM -->|builds| TBIND
    BM -->|links| CUT
    BM -->|links| GTEST
    BM -->|runs| CPPT
    BM -->|runs| PYT

    %% Runtime & Usage
    FA -->|"imports"| TBIND
    SN -->|"imports"| TBIND
    TBIND -->|"invokes"| SK
    KH -->|"defines kernels"| SK
    IH -->|"declares iterators"| SK
    SK -->|"executes on"| CUDA
    SK -->|"uses memory hierarchy"| IH
    E1 -->|"depends on"| IH
    E2 -->|"depends on"| IH
    E3 -->|"depends on"| IH
    BC -->|"benchmarks"| SK
    BP -->|"benchmarks via"| FA

    %% External usage
    TORCH -->|provides| TBIND
    CUDA -->|required by| SK

    %% Click Events
    click BM "https://github.com/microsoft/tilefusion/blob/master/CMakeLists.txt"
    click MK "https://github.com/microsoft/tilefusion/tree/master/Makefile"
    click CM "https://github.com/microsoft/tilefusion/tree/master/cmake/"
    click SP "https://github.com/microsoft/tilefusion/blob/master/setup.py"
    click PP "https://github.com/microsoft/tilefusion/blob/master/pyproject.toml"
    click IH "https://github.com/microsoft/tilefusion/tree/master/include/"
    click CH "https://github.com/microsoft/tilefusion/tree/master/include/cell/"
    click KH "https://github.com/microsoft/tilefusion/tree/master/include/kernels/"
    click TH "https://github.com/microsoft/tilefusion/tree/master/include/types/"
    click UH "https://github.com/microsoft/tilefusion/tree/master/include/util/"
    click TR "https://github.com/microsoft/tilefusion/tree/master/include/traits/"
    click SK "https://github.com/microsoft/tilefusion/tree/master/src/kernels/"
    click CU "https://github.com/microsoft/tilefusion/blob/master/src/cuda_utils.cc"
    click CI "https://github.com/microsoft/tilefusion/blob/master/src/cuda_info.cc"
    click CUH "https://github.com/microsoft/tilefusion/blob/master/include/cuda_utils.hpp"
    click CIH "https://github.com/microsoft/tilefusion/blob/master/include/cuda_info.hpp"
    click KR "https://github.com/microsoft/tilefusion/blob/master/include/kernel_registry.hpp"
    click TBIND "https://github.com/microsoft/tilefusion/blob/master/src/torch_bind.cc"
    click PYI "https://github.com/microsoft/tilefusion/blob/master/python/__init__.py"
    click PYV "https://github.com/microsoft/tilefusion/blob/master/python/__version__.py"
    click FA "https://github.com/microsoft/tilefusion/blob/master/python/ops/flash_attention.py"
    click SN "https://github.com/microsoft/tilefusion/blob/master/python/ops/scatter_nd.py"
    click E1 "https://github.com/microsoft/tilefusion/tree/master/examples/01_gemm/"
    click E2 "https://github.com/microsoft/tilefusion/tree/master/examples/02_fused_two_gemms/"
    click E3 "https://github.com/microsoft/tilefusion/tree/master/examples/03_flash_attention/"
    click BC "https://github.com/microsoft/tilefusion/tree/master/benchmarks/cpp/"
    click BP "https://github.com/microsoft/tilefusion/tree/master/benchmarks/utils/python/"
    click CPPT "https://github.com/microsoft/tilefusion/tree/master/tests/cpp/"
    click PYT "https://github.com/microsoft/tilefusion/tree/master/tests/python/"
    click CUT "https://github.com/microsoft/tilefusion/tree/master/3rd-party/cutlass/"
    click GTEST "https://github.com/microsoft/tilefusion/tree/master/3rd-party/googletest/"

    %% Styles
    classDef cpp fill:#ADD8E6,stroke:#000;
    classDef cuda fill:#87CEFA,stroke:#000;
    classDef py fill:#90EE90,stroke:#000;
    classDef build fill:#FFA500,stroke:#000;
    classDef ext fill:#D3D3D3,stroke:#000;
```
