# philipturner/metal-flash-attention architecture

> Architecture diagram of the GitHub repository philipturner/metal-flash-attention, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/philipturner/metal-flash-attention
- Repository: https://github.com/philipturner/metal-flash-attention

## Overview

##### Project type and purpose
- This is a single‐headed attention library (FlashAttention) implemented in Swift, targeting Apple silicon via Metal.
- It is delivered as a Swift Package. There is no “frontend/UI”—it’s a low‐level compute library for transformer attention kernels (forward and backward).

##### File structure
Top‐level:
- Package.swift / README.md / LICENSE
- Sources/FlashAttention – the core library, subdivided into Attention, GEMM, Utilities
- Tests/FlashAttentionTests – unit tests against Swift, MSL kernels, MTLContext
- Documentation/… – design notes, C++ reference port, archived Swift tests
Key patterns:
- Each “Descriptor” subdirectory holds parameter‐packing and pipeline‐cache logic
- Each “Kernel” subdirectory holds JIT‐compiled multi‐line Swift strings that become Metal Shaders
- Utilities provides MTLContext, address‐space helpers, and cross‐cutting types

##### Extract architecture hints from the README
- The library JIT‐compiles Metal shaders at runtime (no pre‐compiled .metallib).
- Two major algorithmic domains: Attention and GEMM, each with descriptors, kernels, and precision management.

##### Detail & labeling
– Annotate key data structures (AttentionDescriptorParameters, GEMMDescriptor, MTLBuffers)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Subgraph: Client Application
    subgraph "Client Application"
        direction TB
        ClientApp["SwiftPM User Code / Xcode Project"]:::api
    end

    %% Subgraph: Swift API
    subgraph "Swift API" 
        direction TB
        subgraph "Attention Module" 
            direction TB
            AttentionDescriptor["AttentionDescriptor"]:::api
            AttentionKernel["AttentionKernel"]:::api
        end
        subgraph "GEMM Module" 
            direction TB
            GEMMDescriptor["GEMMDescriptor"]:::api
            GEMMKernel["GEMMKernel"]:::api
        end
    end

    %% Utilities & JIT
    subgraph "Support & Runtime Compilation"
        direction TB
        Utilities["Utilities (MTLContext, AddressSpace)"]:::util
        PipelineCache["Pipeline Cache"]:::comp
        MSLStrings["MSL Source Strings"]:::comp
    end

    %% GPU
    subgraph "GPU Execution"
        direction TB
        MTLContext["MTLContext / MTLCompiler"]:::comp
        GPUDevice["GPU (Apple Silicon)"]:::gpu
        GPUBuffer["MTLBuffers"]:::gpu
    end

    %% Test Harness
    subgraph "Test Harness (XCTest)"
        direction TB
        TestHarness["FlashAttentionTests"]:::test
    end

    %% Connections
    ClientApp -->|"builds descriptor"| AttentionDescriptor
    AttentionDescriptor -->|"descriptor parameters"| AttentionKernel
    AttentionDescriptor -->|"uses cache"| PipelineCache
    GEMMDescriptor -->|"descriptor parameters"| GEMMKernel
    GEMMDescriptor -->|"uses cache"| PipelineCache
    AttentionKernel -->|"generates MSL string"| MSLStrings
    GEMMKernel -->|"generates MSL string"| MSLStrings
    MSLStrings -->|"compile()"| MTLContext
    Utilities --> MTLContext
    PipelineCache --> MTLContext
    MTLContext -->|"makeComputePipeline(descriptor:)"| GPUDevice
    GPUDevice -->|"dispatchThreads"| GPUBuffer
    GPUBuffer -->|"results"| ClientApp

    TestHarness -->|"invokes API"| AttentionDescriptor
    TestHarness -->|"invokes API"| GEMMDescriptor
    TestHarness -->|"validates results"| GPUBuffer

    %% Click Events
    click AttentionDescriptor "https://github.com/philipturner/metal-flash-attention/tree/main/Sources/FlashAttention/Attention/AttentionDescriptor"
    click AttentionKernel "https://github.com/philipturner/metal-flash-attention/tree/main/Sources/FlashAttention/Attention/AttentionKernel"
    click GEMMDescriptor "https://github.com/philipturner/metal-flash-attention/tree/main/Sources/FlashAttention/GEMM/GEMMDescriptor"
    click GEMMKernel "https://github.com/philipturner/metal-flash-attention/tree/main/Sources/FlashAttention/GEMM/GEMMKernel"
    click Utilities "https://github.com/philipturner/metal-flash-attention/tree/main/Sources/FlashAttention/Utilities"
    click TestHarness "https://github.com/philipturner/metal-flash-attention/tree/main/Tests/FlashAttentionTests"

    %% Styles
    classDef api fill:#add8e6,stroke:#000,stroke-width:1px
    classDef comp fill:#90ee90,stroke:#000,stroke-width:1px
    classDef gpu fill:#ffa500,stroke:#000,stroke-width:1px
    classDef util fill:#d3d3d3,stroke:#000,stroke-width:1px
    classDef test fill:#eee8aa,stroke:#000,stroke-width:1px
```
