How zml/zml works
ZML is a production inference stack that compiles models for multiple accelerator families. The README’s principal workflow is to provide a model and input, compile for a selected platform, load weights, and execute inference; it also documents local, Hugging Face, and S3 model sources. The supplied tree and excerpts chiefly expose MLIR dialect support, a Trainium attention kernel, profiling, and TPU metrics—not the model CLI or runtime orchestration—so those documented workflow links are not presented as inspected call paths.
Main entry points: CuteInference.cpp CuTe inference · tpuinfo.cc TPU metrics client · traceme_lib.cc TraceMe backend · capi.cc CuTe NVGPU API
Components 21
MLIR dialect support
- CuTe inference
mlir/dialects/cute_ir/CuteInference.cpp - CuTe NVGPU API
mlir/dialects/cute_nvgpu/capi.cc - Fly dialect API
mlir/dialects/fly/capi.cc - TCL operations
mlir/dialects/tcl/tcl_ops.cc - AIR dialect
mlir/dialects/air/AIRDialect.cpp - NVVM inference
mlir/dialects/nvvm/NVVMInference.cpp - CUDA inference
mlir/dialects/cuda/CudaInference.cpp - CUDA Tile API
mlir/dialects/cuda_tile - Mosaic TPU API
mlir/dialects/mosaic_tpu - TTIR API
mlir/dialects/ttir - ROCDL API
mlir/dialects/rocdl/capi.cc - Fly ROCDL API
mlir/dialects/fly_rocdl
Accelerator support
- NKI compiler bridge
platforms/neuron/nki_kernel_compiler.py - TPU metrics client
bin/zml-smi/platforms/tpu/tpuinfo.cc
Observability and metrics
- TraceMe API
zml/profiling/traceme.h - TraceMe backend
zml/profiling/traceme_lib.cc - XProf tool
tools/xprof/main.py
Accelerator kernels
- NKI attention
zml/kernels/flash_attention/nki/attention.py
Other components
- Model user
- Compile, load, infer
- TPU metric service
Connections 6
- Model user → Compile, load, infer: provides model
README.md - Compile, load, infer → CuTe inference: compiles with
README.md - Compile, load, infer → NKI attention: may use
README.md - CuTe inference → CuTe NVGPU API: references dialect
mlir/dialects/cute_ir/CuteInference.cpp - TraceMe backend → TraceMe API: implements API
zml/profiling/traceme_lib.cc - TPU metrics client → TPU metric service: requests metrics
bin/zml-smi/platforms/tpu/tpuinfo.cc