How iree-org/iree-turbine works
Project Type & Purpose
• Open-source Python library/tool
• Bridges PyTorch ⇄ IREE:
– AOT Export of nn.Modules into IREE runtime artifacts
– Eager Execution backend for torch.compile
– Custom-op support
1. User Layer
– Examples, notebooks, CLI scripts
2. API Layer (pure Python)
– iree.turbine.aot
– iree.turbine.dynamo
– iree.turbine.importers
– iree.turbine.runtime API (device/launch/invoke)
– iree.turbine.ops templates
– build_tools (release packaging)
3. IR Pipeline Layer
– FX/torch-mlir importer → “torch” dialect MLIR
– Transfroms (iree.turbine.transforms)
– Kernel codegen (iree.turbine.kernel: Boo, Wave, vector)
4. Compiler Backend
– IREE Base Compiler & Runtime (external)
– VM flatbuffer emitter
5. Execution Layer
– Host runtime (Python wrappers)
– Device runtimes (CPU, ROCm, Vulkan…)
– Op registry, streams, tracing
Main Components & Interactions
• User Script
→ calls AOT Export API or torch.compile(backend="turbine")
• AOT Export
→ FX-based graph extraction
→ Exporter builds CompiledModule
→ Invokes torch-mlir importer
• Importer & IR Transforms
→ produce Torch dialect MLIR
→ iree.turbine.transforms apply metadata, custom-op expansion
• Kernel Codegen
→ selects Boo or Wave codegen paths
→ emits specialized kernels (Jinja, procedural emitter)
• IREE Base Compiler
→ ingests MLIR (–iree-input-type=torch)
→ produces device-specific VM module (.vmfb)
• Runtime Execution
→ Python runtime.invoke / runtime.launch
→ device.py manages contexts/streams
→ op_reg modules register compiled kernels
• Eager Dynamo Backend
→ hooks into torch.compile
→ maps PyTorch ops to IREE/Turbine runtime APIs
Architectural Patterns & Principles
– Layered pipeline: clear separation of API, IR pipeline, compilation, execution
– Plugin/Backend pattern: custom op support via templates + op_reg
– FX graph transformation + MLIR dialect staging
– Modular codegen: Boo vs Wave vs vector backends