# apache/mxnet architecture

> Architecture diagram of the GitHub repository apache/mxnet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apache/mxnet
- Repository: https://github.com/apache/mxnet

## Overview

#### Purpose and Entry Points
Apache MXNet is a C++ deep-learning runtime supporting imperative tensor execution, symbolic graphs, automatic differentiation, and distributed training. The build root is `CMakeLists.txt`, producing the native library exposed through C APIs and language bindings. Primary user entry points are the Python package (`python/mxnet/__init__.py`), C API (`include/mxnet/c_api.h`, `src/c_api/c_api.cc`), and C++ wrapper (`cpp-package/include/mxnet-cpp/MxNetCpp.h`).

#### Tensor Runtime, Memory, and Scheduling
`NDArray` is the core tensor abstraction, with allocation, device placement, and operations implemented in `include/mxnet/ndarray.h`, `src/ndarray/ndarray.cc`, and `src/ndarray/ndarray_function.cc`. Storage managers handle CPU, GPU, pinned, pooled, and shared memory (`src/storage/storage.cc`, `src/storage/gpu_device_storage.h`, `src/storage/pooled_storage_manager.h`). The dependency engine schedules asynchronous read/write tasks and streams across devices, central to both imperative and graph execution (`include/mxnet/engine.h`, `src/engine/engine.cc`, `src/engine/threaded_engine_perdevice.cc`).

#### Frontend and FFI Boundary
Python APIs build tensor/symbol operations and cross into the native runtime through ctypes/Cython FFI layers (`python/mxnet/ndarray/ndarray.py`, `python/mxnet/symbol/symbol.py`, `python/mxnet/_ffi/function.py`). Gluon provides the high-level model, parameter, layer, and training interfaces (`python/mxnet/gluon/block.py`, `python/mxnet/gluon/nn/conv_layers.py`, `python/mxnet/gluon/trainer.py`). This boundary maps Python objects and operator calls onto registered C++ functions; native API registration is defined under `include/mxnet/api_registry.h` and `src/api/_api_internal/_api_internal.cc`.

#### Execution Modes and Graph Optimization
Imperative execution and autograd feed operators directly into engine-scheduled computation; cached/hybrid execution converts repeated workloads to optimized graphs. Core implementation is in `src/imperative/imperative.cc`, `src/imperative/cached_op.cc`, and `src/imperative/infer_graph_attr_pass.cc`. Optimization stages attach execution/resources, infer shapes/types, detect in-place updates, eliminate common expressions, fuse pointwise work, and partition graphs. NNVM graph utilities and memory planning underpin symbolic execution (`src/nnvm/gradient.cc`, `src/nnvm/plan_memory.cc`).

#### Operator and Accelerator Backends
C++ operator registration and implementations form the compute layer, spanning generic tensors, neural networks, NumPy-compatible APIs, sparse/quantized paths, and CPU/CUDA kernels. Representative paths are `src/operator/operator.cc`, `src/operator/nn/convolution.cc`, and `src/operator/numpy/np_matmul_op.cc`. CUDA, cuDNN, and runtime compilation support reside in `src/common/cuda/rtc.cc` and `src/operator/nn/cudnn/cudnn_algoreg.cc`. oneDNN supplies optimized CPU kernels and graph subgraphs (`src/operator/nn/dnnl/dnnl_convolution.cc`, `src/operator/subgraph/dnnl/dnnl_subgraph_property.cc`). TensorRT integration translates selected subgraphs through ONNX (`src/operator/subgraph/tensorrt/onnx_to_tensorrt.cc`).

#### Training, Data, and Distributed Coordination
Autograd records and differentiates imperative operations through the Python frontend and native operator graph. Data ingestion supports RecordIO, images, CSV, LibSVM, batching, and prefetching (`src/io/io.cc`, `src/io/iter_image_recordio.cc`, `src/io/dataloader.cc`). KVStore coordinates parameter/gradient exchange locally or across workers; distributed modes use ps-lite and can use NCCL, Horovod, or BytePS (`src/kvstore/kvstore.cc`, `src/kvstore/kvstore_dist.h`, `src/kvstore/kvstore_nccl.h`; `python/mxnet/kvstore/horovod.py`).

#### Extensibility, Build, and External Dependencies
Native library extensions, custom operators, and subgraph partitioners are supported without forking core runtime (`include/mxnet/lib_api.h`, `src/lib_api.cc`, `src/operator/subgraph/subgraph_property.h`). CMake conditionally selects BLAS, CUDA/cuDNN, NCCL, TensorRT, and oneDNN (`cmake/ChooseBlas.cmake`, `cmake/Modules/FindCUDNN.cmake`, `cmake/BuildTVM.cmake`). Vendored foundations include DLPack, mshadow, TVM, oneDNN, ps-lite, and ONNX-TensorRT under `3rdparty/`.

Main entry points: [`include/mxnet/api_registry.h`](https://github.com/apache/mxnet/blob/master/include/mxnet/api_registry.h) (API registry), [`include/mxnet/ndarray.h`](https://github.com/apache/mxnet/blob/master/include/mxnet/ndarray.h) (NDArray), [`src/operator/operator.cc`](https://github.com/apache/mxnet/blob/master/src/operator/operator.cc) (Operator registry & kernels), [`CMakeLists.txt`](https://github.com/apache/mxnet/blob/master/CMakeLists.txt) (CMake build)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["User APIs"]
  node_python["Python package<br/>Python API<br/>[__init__.py]"]
  node_gluon["Gluon<br/>high-level training API<br/>[block.py]"]
  node_ffi["Python FFI<br/>ctypes/Cython bridge<br/>[function.py]"]
  node_capi["C API<br/>native ABI<br/>[c_api.h]"]
  node_cppapi["C++ wrapper<br/>C++ API<br/>[MxNetCpp.h]"]
end

subgraph group_runtime["Native Runtime"]
  node_registry["API registry<br/>native function registry<br/>[api_registry.h]"]
  node_ndarray["NDArray<br/>tensor runtime<br/>[ndarray.h]"]
  node_storage[("Storage managers<br/>device memory<br/>[storage.cc]")]
  node_engine["Dependency engine<br/>async scheduler<br/>[engine.h]"]
end

subgraph group_execution["Execution &amp; Training"]
  node_imperative["Imperative &amp; autograd<br/>eager execution<br/>[imperative.cc]"]
  node_cachedop["Cached/hybrid execution<br/>graph compiler<br/>[cached_op.cc]"]
  node_nnvm["NNVM graph passes<br/>graph optimization<br/>[plan_memory.cc]"]
  node_operators["Operator registry &amp; kernels<br/>compute layer<br/>[operator.cc]"]
  node_data["Data ingestion<br/>IO pipeline<br/>[dataloader.cc]"]
  node_kvstore["KVStore<br/>parameter synchronization<br/>[kvstore.cc]"]
end

subgraph group_backends["Backends &amp; Integration"]
  node_build["CMake build<br/>build system<br/>[CMakeLists.txt]"]
  node_accelerators["CUDA/cuDNN backend<br/>GPU backend<br/>[rtc.cc]"]
  node_subgraphs["oneDNN &amp; TensorRT<br/>subgraph backend"]
  node_extensions["Extension API<br/>plugin boundary<br/>[lib_api.h]"]
  node_thirdparty["Vendored foundations<br/>third-party runtimes"]
end

node_build -->|"configures"| node_thirdparty
node_build -.->|"enables"| node_accelerators
node_build -.->|"enables"| node_subgraphs
node_python -->|"exposes"| node_gluon
node_gluon -->|"invokes"| node_ffi
node_python -->|"tensor and symbol calls"| node_ffi
node_ffi -->|"crosses ABI"| node_capi
node_cppapi -->|"wraps"| node_capi
node_capi -->|"dispatches"| node_registry
node_registry -->|"creates and operates on"| node_ndarray
node_ndarray -->|"allocates from"| node_storage
node_ndarray -->|"submits dependencies"| node_engine
node_registry -->|"invokes eager execution"| node_imperative
node_imperative -->|"executes"| node_operators
node_imperative -.->|"caches repeated workloads"| node_cachedop
node_cachedop -->|"optimizes graph"| node_nnvm
node_nnvm -->|"lowers execution"| node_operators
node_operators -->|"schedules kernels"| node_engine
node_operators -.->|"uses GPU kernels"| node_accelerators
node_nnvm -.->|"partitions eligible graphs"| node_subgraphs
node_data -->|"produces batches"| node_ndarray
node_gluon -->|"synchronizes training"| node_kvstore
node_kvstore -.->|"uses distributed runtimes"| node_thirdparty
node_extensions -->|"registers functions and operators"| node_registry
node_extensions -.->|"adds partitioners"| node_subgraphs

click node_build "https://github.com/apache/mxnet/blob/master/CMakeLists.txt"
click node_python "https://github.com/apache/mxnet/blob/master/python/mxnet/__init__.py"
click node_gluon "https://github.com/apache/mxnet/blob/master/python/mxnet/gluon/block.py"
click node_ffi "https://github.com/apache/mxnet/blob/master/python/mxnet/_ffi/function.py"
click node_capi "https://github.com/apache/mxnet/blob/master/include/mxnet/c_api.h"
click node_cppapi "https://github.com/apache/mxnet/blob/master/cpp-package/include/mxnet-cpp/MxNetCpp.h"
click node_registry "https://github.com/apache/mxnet/blob/master/include/mxnet/api_registry.h"
click node_ndarray "https://github.com/apache/mxnet/blob/master/include/mxnet/ndarray.h"
click node_storage "https://github.com/apache/mxnet/blob/master/src/storage/storage.cc"
click node_engine "https://github.com/apache/mxnet/blob/master/include/mxnet/engine.h"
click node_imperative "https://github.com/apache/mxnet/blob/master/src/imperative/imperative.cc"
click node_cachedop "https://github.com/apache/mxnet/blob/master/src/imperative/cached_op.cc"
click node_nnvm "https://github.com/apache/mxnet/blob/master/src/nnvm/plan_memory.cc"
click node_operators "https://github.com/apache/mxnet/blob/master/src/operator/operator.cc"
click node_data "https://github.com/apache/mxnet/blob/master/src/io/dataloader.cc"
click node_kvstore "https://github.com/apache/mxnet/blob/master/src/kvstore/kvstore.cc"
click node_accelerators "https://github.com/apache/mxnet/blob/master/src/common/cuda/rtc.cc"
click node_subgraphs "https://github.com/apache/mxnet/blob/master/src/operator/subgraph/dnnl/dnnl_subgraph_property.cc"
click node_extensions "https://github.com/apache/mxnet/blob/master/include/mxnet/lib_api.h"
click node_thirdparty "https://github.com/apache/mxnet/tree/master/3rdparty"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_python,node_gluon,node_ffi,node_capi,node_cppapi toneBlue
class node_registry,node_ndarray,node_storage,node_engine toneAmber
class node_imperative,node_cachedop,node_nnvm,node_operators,node_data,node_kvstore toneMint
class node_build,node_accelerators,node_subgraphs,node_extensions,node_thirdparty toneRose
```

## Components

### User APIs

- [Python package](https://github.com/apache/mxnet/blob/master/python/mxnet/__init__.py): Python API · `python/mxnet/__init__.py`
- [Gluon](https://github.com/apache/mxnet/blob/master/python/mxnet/gluon/block.py): high-level training API · `python/mxnet/gluon/block.py`
- [Python FFI](https://github.com/apache/mxnet/blob/master/python/mxnet/_ffi/function.py): ctypes/Cython bridge · `python/mxnet/_ffi/function.py`
- [C API](https://github.com/apache/mxnet/blob/master/include/mxnet/c_api.h): native ABI · `include/mxnet/c_api.h`
- [C++ wrapper](https://github.com/apache/mxnet/blob/master/cpp-package/include/mxnet-cpp/MxNetCpp.h): C++ API · `cpp-package/include/mxnet-cpp/MxNetCpp.h`

### Native Runtime

- [API registry](https://github.com/apache/mxnet/blob/master/include/mxnet/api_registry.h): native function registry · `include/mxnet/api_registry.h`
- [NDArray](https://github.com/apache/mxnet/blob/master/include/mxnet/ndarray.h): tensor runtime · Owns tensor allocation, placement, and tensor operations. · `include/mxnet/ndarray.h`
- [Storage managers](https://github.com/apache/mxnet/blob/master/src/storage/storage.cc): device memory · Provides CPU, GPU, pinned, pooled, and shared allocations. · `src/storage/storage.cc`
- [Dependency engine](https://github.com/apache/mxnet/blob/master/include/mxnet/engine.h): async scheduler · Schedules dependent read/write tasks and device streams. · `include/mxnet/engine.h`

### Execution & Training

- [Imperative & autograd](https://github.com/apache/mxnet/blob/master/src/imperative/imperative.cc): eager execution · `src/imperative/imperative.cc`
- [Cached/hybrid execution](https://github.com/apache/mxnet/blob/master/src/imperative/cached_op.cc): graph compiler · Converts repeated imperative workloads into optimized graphs. · `src/imperative/cached_op.cc`
- [NNVM graph passes](https://github.com/apache/mxnet/blob/master/src/nnvm/plan_memory.cc): graph optimization · Supplies gradient and memory-planning graph utilities. · `src/nnvm/plan_memory.cc`
- [Operator registry & kernels](https://github.com/apache/mxnet/blob/master/src/operator/operator.cc): compute layer · Implements generic, neural-network, NumPy, sparse, and quantized operators. · `src/operator/operator.cc`
- [Data ingestion](https://github.com/apache/mxnet/blob/master/src/io/dataloader.cc): IO pipeline · Loads records and datasets with batching and prefetching. · `src/io/dataloader.cc`
- [KVStore](https://github.com/apache/mxnet/blob/master/src/kvstore/kvstore.cc): parameter synchronization · Coordinates local and distributed gradient exchange. · `src/kvstore/kvstore.cc`

### Backends & Integration

- [CMake build](https://github.com/apache/mxnet/blob/master/CMakeLists.txt): build system · Builds the native library and selects optional dependencies. · `CMakeLists.txt`
- [CUDA/cuDNN backend](https://github.com/apache/mxnet/blob/master/src/common/cuda/rtc.cc): GPU backend · Provides CUDA kernels, cuDNN algorithms, and runtime compilation. · `src/common/cuda/rtc.cc`
- [oneDNN & TensorRT](https://github.com/apache/mxnet/blob/master/src/operator/subgraph/dnnl/dnnl_subgraph_property.cc): subgraph backend · Partitions and lowers eligible graphs to optimized CPU or TensorRT backends. · `src/operator/subgraph/dnnl/dnnl_subgraph_property.cc`
- [Extension API](https://github.com/apache/mxnet/blob/master/include/mxnet/lib_api.h): plugin boundary · Supports custom operators, libraries, and subgraph partitioners. · `include/mxnet/lib_api.h`
- [Vendored foundations](https://github.com/apache/mxnet/tree/master/3rdparty): third-party runtimes · Includes TVM, oneDNN, ps-lite, DLPack, and related dependencies. · `3rdparty`

## Connections

- CMake build → Vendored foundations: configures
- CMake build → CUDA/cuDNN backend: enables
- CMake build → oneDNN & TensorRT: enables
- Python package → Gluon: exposes
- Gluon → Python FFI: invokes
- Python package → Python FFI: tensor and symbol calls
- Python FFI → C API: crosses ABI
- C++ wrapper → C API: wraps
- C API → API registry: dispatches
- API registry → NDArray: creates and operates on
- NDArray → Storage managers: allocates from
- NDArray → Dependency engine: submits dependencies
- API registry → Imperative & autograd: invokes eager execution
- Imperative & autograd → Operator registry & kernels: executes
- Imperative & autograd → Cached/hybrid execution: caches repeated workloads
- Cached/hybrid execution → NNVM graph passes: optimizes graph
- NNVM graph passes → Operator registry & kernels: lowers execution
- Operator registry & kernels → Dependency engine: schedules kernels
- Operator registry & kernels → CUDA/cuDNN backend: uses GPU kernels
- NNVM graph passes → oneDNN & TensorRT: partitions eligible graphs
- Data ingestion → NDArray: produces batches
- Gluon → KVStore: synchronizes training
- KVStore → Vendored foundations: uses distributed runtimes
- Extension API → API registry: registers functions and operators
- Extension API → oneDNN & TensorRT: adds partitioners
