# iree-org/iree-turbine architecture

> Architecture diagram of the GitHub repository iree-org/iree-turbine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iree-org/iree-turbine
- Repository: https://github.com/iree-org/iree-turbine

## Overview

##### Project Type & Purpose
- Open-source Python library/tool
- Bridges PyTorch ⇄ IREE:
– AOT Export of nn.Modules into IREE runtime artifacts
– Eager Execution backend for torch.compile
– Custom-op support

1. User Layer
– Examples, notebooks, CLI scripts
2. API Layer (pure Python)
– iree.turbine.aot
– iree.turbine.dynamo
– iree.turbine.importers
– iree.turbine.runtime API (device/launch/invoke)
– iree.turbine.ops templates
– build_tools (release packaging)
3. IR Pipeline Layer
– FX/torch-mlir importer → “torch” dialect MLIR
– Transfroms (iree.turbine.transforms)
– Kernel codegen (iree.turbine.kernel: Boo, Wave, vector)
4. Compiler Backend
– IREE Base Compiler & Runtime (external)
– VM flatbuffer emitter
5. Execution Layer
– Host runtime (Python wrappers)
– Device runtimes (CPU, ROCm, Vulkan…)
– Op registry, streams, tracing

##### Main Components & Interactions
- User Script
→ calls AOT Export API or torch.compile(backend="turbine")
- AOT Export
→ FX-based graph extraction
→ Exporter builds CompiledModule
→ Invokes torch-mlir importer
- Importer & IR Transforms
→ produce Torch dialect MLIR
→ iree.turbine.transforms apply metadata, custom-op expansion
- Kernel Codegen
→ selects Boo or Wave codegen paths
→ emits specialized kernels (Jinja, procedural emitter)
- IREE Base Compiler
→ ingests MLIR (–iree-input-type=torch)
→ produces device-specific VM module (.vmfb)
- Runtime Execution
→ Python runtime.invoke / runtime.launch
→ device.py manages contexts/streams
→ op_reg modules register compiled kernels
- Eager Dynamo Backend
→ hooks into torch.compile
→ maps PyTorch ops to IREE/Turbine runtime APIs

##### Architectural Patterns & Principles
– Layered pipeline: clear separation of API, IR pipeline, compilation, execution
– Plugin/Backend pattern: custom op support via templates + op_reg
– FX graph transformation + MLIR dialect staging
– Modular codegen: Boo vs Wave vs vector backends

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User Layer"
        Examples["Examples & CLI"]:::python
        Interpreter["Interpreter CLI"]:::python
    end

    %% API Layer
    subgraph "API Layer"
        AOTExporter["AOT Export API"]:::python
        FXExtract["FX-Based Graph Extraction"]:::python
        Dynamo["Eager Dynamo Backend"]:::python
        Importers["Torch-MLIR Importer"]:::external
        OpsTemplates["Ops Templates"]:::python
        BuildTools["Build & Release Tools"]:::python
        RuntimeAPI["Runtime API (device/launch/invoke)"]:::python
    end

    %% IR Pipeline Layer
    subgraph "IR Pipeline Layer"
        Transforms["IR Transforms"]:::ir
        MLIR["Torch Dialect MLIR"]:::ir
    end

    %% Kernel Codegen Layer
    subgraph "Kernel Codegen"
        Boo["Boo Backend"]:::python
        Wave["Wave Backend"]:::python
        Vector["Vector Backend"]:::python
    end

    %% Compiler Backend
    subgraph "Compiler Backend"
        TorchMLIR["Torch-MLIR"]:::external
        IREEComp["IREE Base Compiler"]:::external
        VMFB["VM FlatBuffer Module"]:::ir
    end

    %% Execution Layer
    subgraph "Execution Layer"
        HostRT["Host Runtime (Python Wrappers)"]:::runtime
        DeviceRT["Device Runtimes"]:::runtime
        OpReg["Op Registry & Extensions"]:::runtime
        Tracing["Runtime Tracing"]:::runtime
    end

    %% Connections
    Examples -->|calls| AOTExporter
    Interpreter -->|calls| AOTExporter
    Examples -->|calls| Dynamo

    AOTExporter -->|uses FX| FXExtract
    FXExtract -->|imports| Importers
    Importers -->|produces MLIR| MLIR
    MLIR -->|transforms| Transforms
    Transforms -->|outputs MLIR| MLIR

    MLIR -->|codegen| Boo
    MLIR -->|codegen| Wave
    MLIR -->|codegen| Vector

    Boo -->|invokes| TorchMLIR
    Wave -->|invokes| TorchMLIR
    Vector -->|invokes| TorchMLIR

    TorchMLIR -->|compiles| IREEComp
    IREEComp -->|emits| VMFB

    VMFB -->|loaded by| RuntimeAPI
    Dynamo -->|uses| RuntimeAPI
    RuntimeAPI -->|launch/invoke| HostRT
    HostRT -->|manages| DeviceRT
    HostRT -->|registers ops| OpReg
    HostRT -->|traces| Tracing

    %% Click Events
    click Examples "https://github.com/iree-org/iree-turbine/tree/main/examples/"
    click Interpreter "https://github.com/iree-org/iree-turbine/blob/main/iree/turbine/tools/interpreter.py"
    click AOTExporter "https://github.com/iree-org/iree-turbine/blob/main/iree/turbine/aot/exporter.py"
    click FXExtract "https://github.com/iree-org/iree-turbine/blob/main/iree/turbine/aot/fx_programs.py"
    click Importers "https://github.com/iree-org/iree-turbine/blob/main/iree/turbine/importers/ir.py"
    click Transforms "https://github.com/iree-org/iree-turbine/tree/main/iree/turbine/transforms/"
    click Boo "https://github.com/iree-org/iree-turbine/tree/main/iree/turbine/kernel/boo/"
    click Wave "https://github.com/iree-org/iree-turbine/tree/main/iree/turbine/kernel/wave/"
    click Vector "https://github.com/iree-org/iree-turbine/blob/main/iree/turbine/kernel/compiler/vector_codegen.py"
    click Dynamo "https://github.com/iree-org/iree-turbine/tree/main/iree/turbine/dynamo/"
    click RuntimeAPI "https://github.com/iree-org/iree-turbine/blob/main/iree/turbine/runtime/device.py"
    click OpReg "https://github.com/iree-org/iree-turbine/tree/main/iree/turbine/runtime/op_reg/"
    click Tracing "https://github.com/iree-org/iree-turbine/blob/main/iree/turbine/runtime/tracing.py"
    click OpsTemplates "https://github.com/iree-org/iree-turbine/tree/main/iree/turbine/ops/templates/"
    click BuildTools "https://github.com/iree-org/iree-turbine/tree/main/build_tools/"

    %% Styles
    classDef python fill:#D0E1FF,stroke:#0366d6,color:#0366d6
    classDef ir fill:#DFFFD0,stroke:#6A8A00,color:#6A8A00
    classDef external fill:#F0D0FF,stroke:#8A2BE2,color:#8A2BE2
    classDef runtime fill:#FFE5B4,stroke:#FF8C00,color:#FF8C00
```
