How apache/mxnet works
Purpose and Entry Points
Apache MXNet is a C++ deep-learning runtime supporting imperative tensor execution, symbolic graphs, automatic differentiation, and distributed training. The build root is CMakeLists.txt, producing the native library exposed through C APIs and language bindings. Primary user entry points are the Python package (python/mxnet/__init__.py), C API (include/mxnet/c_api.h, src/c_api/c_api.cc), and C++ wrapper (cpp-package/include/mxnet-cpp/MxNetCpp.h).
Tensor Runtime, Memory, and Scheduling
NDArray is the core tensor abstraction, with allocation, device placement, and operations implemented in include/mxnet/ndarray.h, src/ndarray/ndarray.cc, and src/ndarray/ndarray_function.cc. Storage managers handle CPU, GPU, pinned, pooled, and shared memory (src/storage/storage.cc, src/storage/gpu_device_storage.h, src/storage/pooled_storage_manager.h). The dependency engine schedules asynchronous read/write tasks and streams across devices, central to both imperative and graph execution (include/mxnet/engine.h, src/engine/engine.cc, src/engine/threaded_engine_perdevice.cc).
Frontend and FFI Boundary
Python APIs build tensor/symbol operations and cross into the native runtime through ctypes/Cython FFI layers (python/mxnet/ndarray/ndarray.py, python/mxnet/symbol/symbol.py, python/mxnet/_ffi/function.py). Gluon provides the high-level model, parameter, layer, and training interfaces (python/mxnet/gluon/block.py, python/mxnet/gluon/nn/conv_layers.py, python/mxnet/gluon/trainer.py). This boundary maps Python objects and operator calls onto registered C++ functions; native API registration is defined under include/mxnet/api_registry.h and src/api/_api_internal/_api_internal.cc.
Execution Modes and Graph Optimization
Imperative execution and autograd feed operators directly into engine-scheduled computation; cached/hybrid execution converts repeated workloads to optimized graphs. Core implementation is in src/imperative/imperative.cc, src/imperative/cached_op.cc, and src/imperative/infer_graph_attr_pass.cc. Optimization stages attach execution/resources, infer shapes/types, detect in-place updates, eliminate common expressions, fuse pointwise work, and partition graphs. NNVM graph utilities and memory planning underpin symbolic execution (src/nnvm/gradient.cc, src/nnvm/plan_memory.cc).
Operator and Accelerator Backends
C++ operator registration and implementations form the compute layer, spanning generic tensors, neural networks, NumPy-compatible APIs, sparse/quantized paths, and CPU/CUDA kernels. Representative paths are src/operator/operator.cc, src/operator/nn/convolution.cc, and src/operator/numpy/np_matmul_op.cc. CUDA, cuDNN, and runtime compilation support reside in src/common/cuda/rtc.cc and src/operator/nn/cudnn/cudnn_algoreg.cc. oneDNN supplies optimized CPU kernels and graph subgraphs (src/operator/nn/dnnl/dnnl_convolution.cc, src/operator/subgraph/dnnl/dnnl_subgraph_property.cc). TensorRT integration translates selected subgraphs through ONNX (src/operator/subgraph/tensorrt/onnx_to_tensorrt.cc).
Training, Data, and Distributed Coordination
Autograd records and differentiates imperative operations through the Python frontend and native operator graph. Data ingestion supports RecordIO, images, CSV, LibSVM, batching, and prefetching (src/io/io.cc, src/io/iter_image_recordio.cc, src/io/dataloader.cc). KVStore coordinates parameter/gradient exchange locally or across workers; distributed modes use ps-lite and can use NCCL, Horovod, or BytePS (src/kvstore/kvstore.cc, src/kvstore/kvstore_dist.h, src/kvstore/kvstore_nccl.h; python/mxnet/kvstore/horovod.py).
Extensibility, Build, and External Dependencies
Native library extensions, custom operators, and subgraph partitioners are supported without forking core runtime (include/mxnet/lib_api.h, src/lib_api.cc, src/operator/subgraph/subgraph_property.h). CMake conditionally selects BLAS, CUDA/cuDNN, NCCL, TensorRT, and oneDNN (cmake/ChooseBlas.cmake, cmake/Modules/FindCUDNN.cmake, cmake/BuildTVM.cmake). Vendored foundations include DLPack, mshadow, TVM, oneDNN, ps-lite, and ONNX-TensorRT under 3rdparty/.
Main entry points: api_registry.h API registry · ndarray.h NDArray · operator.cc Operator registry & kernels · CMakeLists.txt CMake build
Components 20
User APIs
- Python package
python/mxnet/__init__.py - Gluon
python/mxnet/gluon/block.py - Python FFI
python/mxnet/_ffi/function.py - C API
include/mxnet/c_api.h - C++ wrapper
cpp-package/include/mxnet-cpp/MxNetCpp.h
Native Runtime
- API registry
include/mxnet/api_registry.h - NDArray · Owns tensor allocation, placement, and tensor operations.
include/mxnet/ndarray.h - Storage managers · Provides CPU, GPU, pinned, pooled, and shared allocations.
src/storage/storage.cc - Dependency engine · Schedules dependent read/write tasks and device streams.
include/mxnet/engine.h
Execution & Training
- Imperative & autograd
src/imperative/imperative.cc - Cached/hybrid execution · Converts repeated imperative workloads into optimized graphs.
src/imperative/cached_op.cc - NNVM graph passes · Supplies gradient and memory-planning graph utilities.
src/nnvm/plan_memory.cc - Operator registry & kernels · Implements generic, neural-network, NumPy, sparse, and quantized operators.
src/operator/operator.cc - Data ingestion · Loads records and datasets with batching and prefetching.
src/io/dataloader.cc - KVStore · Coordinates local and distributed gradient exchange.
src/kvstore/kvstore.cc
Backends & Integration
- CMake build · Builds the native library and selects optional dependencies.
CMakeLists.txt - CUDA/cuDNN backend · Provides CUDA kernels, cuDNN algorithms, and runtime compilation.
src/common/cuda/rtc.cc - oneDNN & TensorRT · Partitions and lowers eligible graphs to optimized CPU or TensorRT backends.
src/operator/subgraph/dnnl/dnnl_subgraph_property.cc - Extension API · Supports custom operators, libraries, and subgraph partitioners.
include/mxnet/lib_api.h - Vendored foundations · Includes TVM, oneDNN, ps-lite, DLPack, and related dependencies.
3rdparty
Connections 25
- CMake build → Vendored foundations: configures
- CMake build → CUDA/cuDNN backend: enables
- CMake build → oneDNN & TensorRT: enables
- Python package → Gluon: exposes
- Gluon → Python FFI: invokes
- Python package → Python FFI: tensor and symbol calls
- Python FFI → C API: crosses ABI
- C++ wrapper → C API: wraps
- C API → API registry: dispatches
- API registry → NDArray: creates and operates on
- NDArray → Storage managers: allocates from
- NDArray → Dependency engine: submits dependencies
- API registry → Imperative & autograd: invokes eager execution
- Imperative & autograd → Operator registry & kernels: executes
- Imperative & autograd → Cached/hybrid execution: caches repeated workloads
- Cached/hybrid execution → NNVM graph passes: optimizes graph
- NNVM graph passes → Operator registry & kernels: lowers execution
- Operator registry & kernels → Dependency engine: schedules kernels
- Operator registry & kernels → CUDA/cuDNN backend: uses GPU kernels
- NNVM graph passes → oneDNN & TensorRT: partitions eligible graphs
- Data ingestion → NDArray: produces batches
- Gluon → KVStore: synchronizes training
- KVStore → Vendored foundations: uses distributed runtimes
- Extension API → API registry: registers functions and operators
- Extension API → oneDNN & TensorRT: adds partitioners