How ml-explore/mlx works
Purpose and entry points
MLX is a C++ tensor/array runtime for ML research, exposing NumPy-like Python APIs plus C++ and C interfaces. It builds lazy, dynamic computation graphs with autodiff, vectorization, compilation, CPU/GPU execution, and unified-memory-oriented device handling. Public C++ surface starts at mlx/mlx.h, arrays at mlx/array.h, and Python native module bindings at python/src/mlx.cpp.
Array graph and lazy execution
array represents tensor values and deferred primitive nodes; public operations construct graph nodes rather than necessarily materializing results. Evaluation, dependency scheduling, streams, events, fences, and device selection form the execution boundary.
• mlx/array.cpp
• mlx/primitives.h
• mlx/scheduler.cpp
Operations, transforms, and compilation
The core operation layer covers elementwise, linalg, FFT, random, einsum, and specialized fast ML operations. Function transforms implement autodiff/custom VJPs and vectorization; compilation provides graph optimization/execution specialization while preserving dynamic graph semantics.
• mlx/ops.cpp
• mlx/transforms.cpp
• mlx/compile.cpp
Backend abstraction and execution
Shared backend code defines reusable operation semantics and compiled-kernel interfaces, while device-specific backends own allocation, evaluation, synchronization, and kernel dispatch. CPU is always represented; Apple Metal and CUDA are alternative accelerators, with no-op backend variants supporting reduced builds.
• mlx/backend/common/compiled.cpp
• mlx/backend/cpu/eval.cpp
• mlx/backend/metal/eval.cpp
• mlx/backend/cuda/eval.cpp
Accelerator kernels and external runtimes
Metal uses host C++ dispatch plus .metal kernels, including tuned Steel GEMM, convolution, and attention paths. CUDA uses .cu kernels and integrates CUDA libraries such as cuBLAS, cuDNN, and cuSOLVER. CPU includes Accelerate/CBLAS-style and SIMD GEMM implementations. CMake selects platform/backend dependencies.
• mlx/backend/metal/metal.cpp
• mlx/backend/metal/kernels/steel/gemm/kernels/steel_gemm_fused.metal
• mlx/backend/cuda/cublas_utils.cpp
• mlx/CMakeLists.txt
Python framework layer
Pybind-style C++ bindings translate Python arrays, indexing, devices, transforms, and operation calls into the native runtime. Pure Python packages add neural-network modules, initialization, losses, optimizers, schedulers, utility tree operations, extension support, and distributed-launch utilities.
• python/src/array.cpp
• python/src/ops.cpp
• python/mlx/nn/layers/base.py
• python/mlx/optimizers/optimizers.py
Distributed and persistence boundaries
Distributed primitives route collectives through selectable JACCL, MPI, NCCL, or ring implementations, with backend-specific device integration. Array persistence/import paths support safetensors and GGUF formats. These are optional build/runtime integrations rather than prerequisites for single-device computation.
• mlx/distributed/distributed.cpp
• mlx/distributed/nccl/nccl.cpp
• mlx/io/safetensors.cpp
• mlx/io/gguf.cpp
Build and delivery
CMake drives the native library, selected CPU/Metal/CUDA/distributed components, examples, and tests. Python packaging (setup.py, pyproject.toml) builds/distributes wheels; GitHub workflows build and test Linux, macOS, Windows, wheels, releases, and documentation.
• CMakeLists.txt
• setup.py
• pyproject.toml
• .github/workflows/build_and_test.yml
Main entry points: CMakeLists.txt Native build selection · compiled.cpp Shared backend layer · array.cpp Array graph · mlx.cpp Python native module
Components 20
Public APIs
- C++ / C API · Public runtime and array entry points.
mlx/mlx.h - Python native module
python/src/mlx.cpp - Python array & ops bindings
python/src/array.cpp - Python ML framework · NN layers, optimizers, and utilities build on native arrays.
python/mlx/nn/layers/base.py
Lazy Runtime
- Array graph · Arrays hold values or deferred primitive nodes.
mlx/array.cpp - Primitives
mlx/primitives.h - Scheduler & synchronization · Schedules dependencies, streams, events, fences, and devices.
mlx/scheduler.cpp - Core operations · Elementwise, linalg, FFT, random, einsum, and ML ops.
mlx/ops.cpp - Autodiff & vectorization · Includes custom VJPs and vectorization.
mlx/transforms.cpp - Graph compilation · Optimizes and specializes dynamic graphs.
mlx/compile.cpp
Execution Backends
- Shared backend layer
mlx/backend/common/compiled.cpp - CPU backend · Always available execution backend.
mlx/backend/cpu/eval.cpp - Metal backend · Apple GPU allocation, evaluation, synchronization, and dispatch.
mlx/backend/metal/eval.cpp - Metal kernels · Includes tuned Steel GEMM paths.
mlx/backend/metal/kernels/steel/gemm/kernels/steel_gemm_fused.metal - CUDA backend · CUDA GPU allocation, evaluation, synchronization, and dispatch.
mlx/backend/cuda/eval.cpp
Optional Integrations
- Distributed collectives · Selects JACCL, MPI, NCCL, or ring collectives.
mlx/distributed/distributed.cpp - Tensor persistence · Safetensors import and persistence.
mlx/io/safetensors.cpp
Build & Delivery
- Native build selection · Selects platform backends and dependencies.
CMakeLists.txt - Python packaging
setup.py - CI & releases · Builds and tests platforms, wheels, releases, and docs.
.github/workflows/build_and_test.yml
Connections 22
- C++ / C API → Array graph: creates / evaluates
- Python native module → C++ / C API: exposes
- Python array & ops bindings → Array graph: maps Python arrays
- Python ML framework → Python native module: uses native runtime
- Python ML framework → Python array & ops bindings: uses operations
- Core operations → Primitives: constructs
- Autodiff & vectorization → Core operations: transforms
- Graph compilation → Array graph: optimizes / specializes
- Array graph → Scheduler & synchronization: evaluation dependencies
- Scheduler & synchronization → Shared backend layer: dispatches
- Shared backend layer → CPU backend: shared interfaces
- Shared backend layer → Metal backend: shared interfaces
- Shared backend layer → CUDA backend: shared interfaces
- Metal backend → Metal kernels: dispatches
- Distributed collectives → CUDA backend: device integration
- Native build selection → CPU backend: builds
- Native build selection → Metal backend: selects optionally
- Native build selection → CUDA backend: selects optionally
- Native build selection → Distributed collectives: selects optionally
- Python packaging → Python native module: builds extension
- CI & releases → Native build selection: builds / tests
- CI & releases → Python packaging: publishes wheels