# ml-explore/mlx architecture

> Architecture diagram of the GitHub repository ml-explore/mlx, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ml-explore/mlx
- Repository: https://github.com/ml-explore/mlx

## Overview

#### Purpose and entry points
MLX is a C++ tensor/array runtime for ML research, exposing NumPy-like Python APIs plus C++ and C interfaces. It builds lazy, dynamic computation graphs with autodiff, vectorization, compilation, CPU/GPU execution, and unified-memory-oriented device handling. Public C++ surface starts at `mlx/mlx.h`, arrays at `mlx/array.h`, and Python native module bindings at `python/src/mlx.cpp`.

#### Array graph and lazy execution
`array` represents tensor values and deferred primitive nodes; public operations construct graph nodes rather than necessarily materializing results. Evaluation, dependency scheduling, streams, events, fences, and device selection form the execution boundary.
- `mlx/array.cpp`
- `mlx/primitives.h`
- `mlx/scheduler.cpp`

#### Operations, transforms, and compilation
The core operation layer covers elementwise, linalg, FFT, random, einsum, and specialized fast ML operations. Function transforms implement autodiff/custom VJPs and vectorization; compilation provides graph optimization/execution specialization while preserving dynamic graph semantics.
- `mlx/ops.cpp`
- `mlx/transforms.cpp`
- `mlx/compile.cpp`

#### Backend abstraction and execution
Shared backend code defines reusable operation semantics and compiled-kernel interfaces, while device-specific backends own allocation, evaluation, synchronization, and kernel dispatch. CPU is always represented; Apple Metal and CUDA are alternative accelerators, with no-op backend variants supporting reduced builds.
- `mlx/backend/common/compiled.cpp`
- `mlx/backend/cpu/eval.cpp`
- `mlx/backend/metal/eval.cpp`
- `mlx/backend/cuda/eval.cpp`

#### Accelerator kernels and external runtimes
Metal uses host C++ dispatch plus `.metal` kernels, including tuned Steel GEMM, convolution, and attention paths. CUDA uses `.cu` kernels and integrates CUDA libraries such as cuBLAS, cuDNN, and cuSOLVER. CPU includes Accelerate/CBLAS-style and SIMD GEMM implementations. CMake selects platform/backend dependencies.
- `mlx/backend/metal/metal.cpp`
- `mlx/backend/metal/kernels/steel/gemm/kernels/steel_gemm_fused.metal`
- `mlx/backend/cuda/cublas_utils.cpp`
- `mlx/CMakeLists.txt`

#### Python framework layer
Pybind-style C++ bindings translate Python arrays, indexing, devices, transforms, and operation calls into the native runtime. Pure Python packages add neural-network modules, initialization, losses, optimizers, schedulers, utility tree operations, extension support, and distributed-launch utilities.
- `python/src/array.cpp`
- `python/src/ops.cpp`
- `python/mlx/nn/layers/base.py`
- `python/mlx/optimizers/optimizers.py`

#### Distributed and persistence boundaries
Distributed primitives route collectives through selectable JACCL, MPI, NCCL, or ring implementations, with backend-specific device integration. Array persistence/import paths support safetensors and GGUF formats. These are optional build/runtime integrations rather than prerequisites for single-device computation.
- `mlx/distributed/distributed.cpp`
- `mlx/distributed/nccl/nccl.cpp`
- `mlx/io/safetensors.cpp`
- `mlx/io/gguf.cpp`

#### Build and delivery
CMake drives the native library, selected CPU/Metal/CUDA/distributed components, examples, and tests. Python packaging (`setup.py`, `pyproject.toml`) builds/distributes wheels; GitHub workflows build and test Linux, macOS, Windows, wheels, releases, and documentation.
- `CMakeLists.txt`
- `setup.py`
- `pyproject.toml`
- `.github/workflows/build_and_test.yml`

Main entry points: [`CMakeLists.txt`](https://github.com/ml-explore/mlx/blob/main/CMakeLists.txt) (Native build selection), [`mlx/backend/common/compiled.cpp`](https://github.com/ml-explore/mlx/blob/main/mlx/backend/common/compiled.cpp) (Shared backend layer), [`mlx/array.cpp`](https://github.com/ml-explore/mlx/blob/main/mlx/array.cpp) (Array graph), [`python/src/mlx.cpp`](https://github.com/ml-explore/mlx/blob/main/python/src/mlx.cpp) (Python native module)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Public APIs"]
  node_cpp_api["C++ / C API<br/>public headers<br/>[mlx.h]"]
  node_python_native["Python native module<br/>pybind bindings<br/>[mlx.cpp]"]
  node_python_array_ops["Python array &amp; ops bindings<br/>binding adapters<br/>[array.cpp]"]
  node_python_framework["Python ML framework<br/>pure Python packages<br/>[base.py]"]
end

subgraph group_runtime["Lazy Runtime"]
  node_array_graph{{"Array graph<br/>lazy tensor IR<br/>[array.cpp]"}}
  node_primitives["Primitives<br/>operation nodes<br/>[primitives.h]"]
  node_scheduler["Scheduler &amp; synchronization<br/>execution control<br/>[scheduler.cpp]"]
  node_ops["Core operations<br/>operation layer<br/>[ops.cpp]"]
  node_transforms["Autodiff &amp; vectorization<br/>function transforms<br/>[transforms.cpp]"]
  node_compile["Graph compilation<br/>specialization<br/>[compile.cpp]"]
end

subgraph group_backends["Execution Backends"]
  node_backend_common["Shared backend layer<br/>compiled-kernel interface<br/>[compiled.cpp]"]
  node_cpu["CPU backend<br/>device evaluator<br/>[eval.cpp]"]
  node_metal["Metal backend<br/>device evaluator<br/>[eval.cpp]"]
  node_metal_kernels["Metal kernels<br/>GPU kernels"]
  node_cuda["CUDA backend<br/>device evaluator<br/>[eval.cpp]"]
end

subgraph group_integrations["Optional Integrations"]
  node_distributed["Distributed collectives<br/>optional communication<br/>[distributed.cpp]"]
  node_persistence["Tensor persistence<br/>optional I/O<br/>[safetensors.cpp]"]
end

subgraph group_delivery["Build &amp; Delivery"]
  node_cmake["Native build selection<br/>CMake build<br/>[CMakeLists.txt]"]
  node_python_package["Python packaging<br/>wheel build<br/>[setup.py]"]
  node_ci["CI &amp; releases<br/>GitHub Actions<br/>[build_and_test.yml]"]
end

node_cpp_api -->|"creates / evaluates"| node_array_graph
node_python_native -->|"exposes"| node_cpp_api
node_python_array_ops -->|"maps Python arrays"| node_array_graph
node_python_framework -->|"uses native runtime"| node_python_native
node_python_framework -->|"uses operations"| node_python_array_ops
node_ops -->|"constructs"| node_primitives
node_transforms -->|"transforms"| node_ops
node_compile -->|"optimizes / specializes"| node_array_graph
node_array_graph -->|"evaluation dependencies"| node_scheduler
node_scheduler -->|"dispatches"| node_backend_common
node_backend_common -->|"shared interfaces"| node_cpu
node_backend_common -->|"shared interfaces"| node_metal
node_backend_common -->|"shared interfaces"| node_cuda
node_metal -->|"dispatches"| node_metal_kernels
node_distributed -.->|"device integration"| node_cuda
node_cmake -->|"builds"| node_cpu
node_cmake -.->|"selects optionally"| node_metal
node_cmake -.->|"selects optionally"| node_cuda
node_cmake -.->|"selects optionally"| node_distributed
node_python_package -->|"builds extension"| node_python_native
node_ci -->|"builds / tests"| node_cmake
node_ci -->|"publishes wheels"| node_python_package

click node_cpp_api "https://github.com/ml-explore/mlx/blob/main/mlx/mlx.h"
click node_python_native "https://github.com/ml-explore/mlx/blob/main/python/src/mlx.cpp"
click node_python_array_ops "https://github.com/ml-explore/mlx/blob/main/python/src/array.cpp"
click node_python_framework "https://github.com/ml-explore/mlx/blob/main/python/mlx/nn/layers/base.py"
click node_array_graph "https://github.com/ml-explore/mlx/blob/main/mlx/array.cpp"
click node_primitives "https://github.com/ml-explore/mlx/blob/main/mlx/primitives.h"
click node_scheduler "https://github.com/ml-explore/mlx/blob/main/mlx/scheduler.cpp"
click node_ops "https://github.com/ml-explore/mlx/blob/main/mlx/ops.cpp"
click node_transforms "https://github.com/ml-explore/mlx/blob/main/mlx/transforms.cpp"
click node_compile "https://github.com/ml-explore/mlx/blob/main/mlx/compile.cpp"
click node_backend_common "https://github.com/ml-explore/mlx/blob/main/mlx/backend/common/compiled.cpp"
click node_cpu "https://github.com/ml-explore/mlx/blob/main/mlx/backend/cpu/eval.cpp"
click node_metal "https://github.com/ml-explore/mlx/blob/main/mlx/backend/metal/eval.cpp"
click node_metal_kernels "https://github.com/ml-explore/mlx/blob/main/mlx/backend/metal/kernels/steel/gemm/kernels/steel_gemm_fused.metal"
click node_cuda "https://github.com/ml-explore/mlx/blob/main/mlx/backend/cuda/eval.cpp"
click node_distributed "https://github.com/ml-explore/mlx/blob/main/mlx/distributed/distributed.cpp"
click node_persistence "https://github.com/ml-explore/mlx/blob/main/mlx/io/safetensors.cpp"
click node_cmake "https://github.com/ml-explore/mlx/blob/main/CMakeLists.txt"
click node_python_package "https://github.com/ml-explore/mlx/blob/main/setup.py"
click node_ci "https://github.com/ml-explore/mlx/blob/main/.github/workflows/build_and_test.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cpp_api,node_python_native,node_python_array_ops,node_python_framework toneBlue
class node_array_graph,node_primitives,node_scheduler,node_ops,node_transforms,node_compile toneAmber
class node_backend_common,node_cpu,node_metal,node_metal_kernels,node_cuda toneMint
class node_distributed,node_persistence toneRose
class node_cmake,node_python_package,node_ci toneIndigo
```

## Components

### Public APIs

- [C++ / C API](https://github.com/ml-explore/mlx/blob/main/mlx/mlx.h): public headers · Public runtime and array entry points. · `mlx/mlx.h`
- [Python native module](https://github.com/ml-explore/mlx/blob/main/python/src/mlx.cpp): pybind bindings · `python/src/mlx.cpp`
- [Python array & ops bindings](https://github.com/ml-explore/mlx/blob/main/python/src/array.cpp): binding adapters · `python/src/array.cpp`
- [Python ML framework](https://github.com/ml-explore/mlx/blob/main/python/mlx/nn/layers/base.py): pure Python packages · NN layers, optimizers, and utilities build on native arrays. · `python/mlx/nn/layers/base.py`

### Lazy Runtime

- [Array graph](https://github.com/ml-explore/mlx/blob/main/mlx/array.cpp): lazy tensor IR · Arrays hold values or deferred primitive nodes. · `mlx/array.cpp`
- [Primitives](https://github.com/ml-explore/mlx/blob/main/mlx/primitives.h): operation nodes · `mlx/primitives.h`
- [Scheduler & synchronization](https://github.com/ml-explore/mlx/blob/main/mlx/scheduler.cpp): execution control · Schedules dependencies, streams, events, fences, and devices. · `mlx/scheduler.cpp`
- [Core operations](https://github.com/ml-explore/mlx/blob/main/mlx/ops.cpp): operation layer · Elementwise, linalg, FFT, random, einsum, and ML ops. · `mlx/ops.cpp`
- [Autodiff & vectorization](https://github.com/ml-explore/mlx/blob/main/mlx/transforms.cpp): function transforms · Includes custom VJPs and vectorization. · `mlx/transforms.cpp`
- [Graph compilation](https://github.com/ml-explore/mlx/blob/main/mlx/compile.cpp): specialization · Optimizes and specializes dynamic graphs. · `mlx/compile.cpp`

### Execution Backends

- [Shared backend layer](https://github.com/ml-explore/mlx/blob/main/mlx/backend/common/compiled.cpp): compiled-kernel interface · `mlx/backend/common/compiled.cpp`
- [CPU backend](https://github.com/ml-explore/mlx/blob/main/mlx/backend/cpu/eval.cpp): device evaluator · Always available execution backend. · `mlx/backend/cpu/eval.cpp`
- [Metal backend](https://github.com/ml-explore/mlx/blob/main/mlx/backend/metal/eval.cpp): device evaluator · Apple GPU allocation, evaluation, synchronization, and dispatch. · `mlx/backend/metal/eval.cpp`
- [Metal kernels](https://github.com/ml-explore/mlx/blob/main/mlx/backend/metal/kernels/steel/gemm/kernels/steel_gemm_fused.metal): GPU kernels · Includes tuned Steel GEMM paths. · `mlx/backend/metal/kernels/steel/gemm/kernels/steel_gemm_fused.metal`
- [CUDA backend](https://github.com/ml-explore/mlx/blob/main/mlx/backend/cuda/eval.cpp): device evaluator · CUDA GPU allocation, evaluation, synchronization, and dispatch. · `mlx/backend/cuda/eval.cpp`

### Optional Integrations

- [Distributed collectives](https://github.com/ml-explore/mlx/blob/main/mlx/distributed/distributed.cpp): optional communication · Selects JACCL, MPI, NCCL, or ring collectives. · `mlx/distributed/distributed.cpp`
- [Tensor persistence](https://github.com/ml-explore/mlx/blob/main/mlx/io/safetensors.cpp): optional I/O · Safetensors import and persistence. · `mlx/io/safetensors.cpp`

### Build & Delivery

- [Native build selection](https://github.com/ml-explore/mlx/blob/main/CMakeLists.txt): CMake build · Selects platform backends and dependencies. · `CMakeLists.txt`
- [Python packaging](https://github.com/ml-explore/mlx/blob/main/setup.py): wheel build · `setup.py`
- [CI & releases](https://github.com/ml-explore/mlx/blob/main/.github/workflows/build_and_test.yml): GitHub Actions · Builds and tests platforms, wheels, releases, and docs. · `.github/workflows/build_and_test.yml`

## Connections

- C++ / C API → Array graph: creates / evaluates
- Python native module → C++ / C API: exposes
- Python array & ops bindings → Array graph: maps Python arrays
- Python ML framework → Python native module: uses native runtime
- Python ML framework → Python array & ops bindings: uses operations
- Core operations → Primitives: constructs
- Autodiff & vectorization → Core operations: transforms
- Graph compilation → Array graph: optimizes / specializes
- Array graph → Scheduler & synchronization: evaluation dependencies
- Scheduler & synchronization → Shared backend layer: dispatches
- Shared backend layer → CPU backend: shared interfaces
- Shared backend layer → Metal backend: shared interfaces
- Shared backend layer → CUDA backend: shared interfaces
- Metal backend → Metal kernels: dispatches
- Distributed collectives → CUDA backend: device integration
- Native build selection → CPU backend: builds
- Native build selection → Metal backend: selects optionally
- Native build selection → CUDA backend: selects optionally
- Native build selection → Distributed collectives: selects optionally
- Python packaging → Python native module: builds extension
- CI & releases → Native build selection: builds / tests
- CI & releases → Python packaging: publishes wheels
