How ai-dynamo/nixl works
NIXL is a modular library for accelerating point-to-point inference data transfers across CPU/GPU memory and storage. The principal workflow is an application using the C++ or Python API to describe buffers and request transfers; a plugin-backed engine performs the operation and exposes progress/status. The repository also includes metadata distribution, telemetry, and tracing. The README documents the plugin abstraction and optional ETCD coordination; sampled source confirms backend interfaces, plugin management, and selected engine behavior. Many plugin implementations were not sampled, so their connections are documented capabilities rather than inspected call paths.
Main entry points: backend_engine.h Backend interface · nixl_agent.cpp NIXL agent · nixl.h C++ API · nixl_md_manager.cpp Metadata manager
Components 22
Application APIs
- C++ API
src/api/cpp/nixl.h - Python API
src/api/python/_api.py - Buffer descriptors
src/api/cpp/nixl_descriptors.h
Transfer Core
- NIXL agent
src/core/nixl_agent.cpp - Plugin manager
src/core/nixl_plugin_manager.cpp - Backend interface
src/api/cpp/backend/backend_engine.h
Transfer Engines
- UCX transport
src/plugins/ucx/ucx_thread_pool_engine.cpp - Libfabric transport
src/utils/libfabric/libfabric_rail_manager.cpp - GPU direct storage
src/plugins/cuda_gds/gds_batch_engine.cpp - Object storage
src/plugins/obj/s3/engine_impl.cpp - Object engine registry
src/plugins/obj/obj_engine_registry.h - POSIX storage
src/plugins/posix/posix_backend.cpp - Azure Blob
src/plugins/azure_blob/azure_blob_backend.cpp
Coordination
- Metadata manager
src/core/nixl_md_manager.cpp - Peer metadata
src/core/nixl_p2p_metadata_backend.cpp - ETCD metadata
src/core/nixl_etcd_metadata_backend.cpp
Observability
- Telemetry core
src/core/telemetry/telemetry.cpp - Telemetry exporters
src/plugins/telemetry/prometheus/prometheus_exporter.cpp - Tracing core
src/core/tracing/tracer.cpp - NVTX tracing
src/plugins/tracing/nvtx/nvtx_trace_backend.cpp
Other components
- Inference application
- ETCD service
Connections 22
- Inference application → C++ API: calls
- Inference application → Python API: calls
- C++ API → NIXL agent: submits requests
- Python API → NIXL agent: submits requests
- C++ API → Buffer descriptors: uses buffers
src/api/cpp/nixl.h - NIXL agent → Plugin manager: loads plugins
- Plugin manager → Backend interface: creates engines
src/core/nixl_plugin_manager.cpp - Backend interface → UCX transport: dispatches transfers
- Backend interface → Libfabric transport: dispatches transfers
- Backend interface → GPU direct storage: dispatches transfers
- Backend interface → Object storage: dispatches transfers
- Backend interface → POSIX storage: dispatches transfers
- Backend interface → Azure Blob: dispatches transfers
- Object storage → Object engine registry: registers engine
- NIXL agent → Metadata manager: coordinates metadata
- Metadata manager → Peer metadata: selects backend
src/core/nixl_md_manager.cpp - Metadata manager → ETCD metadata: selects backend
src/core/nixl_md_manager.cpp - ETCD metadata → ETCD service: distributes metadata
README.md - NIXL agent → Telemetry core: records events
- Telemetry core → Telemetry exporters: exports metrics
- NIXL agent → Tracing core: emits traces
- Tracing core → NVTX tracing: uses backend