# ai-dynamo/nixl architecture

> Architecture diagram of the GitHub repository ai-dynamo/nixl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ai-dynamo/nixl
- Repository: https://github.com/ai-dynamo/nixl

## Overview

NIXL is a modular library for accelerating point-to-point inference data transfers across CPU/GPU memory and storage. The principal workflow is an application using the C++ or Python API to describe buffers and request transfers; a plugin-backed engine performs the operation and exposes progress/status. The repository also includes metadata distribution, telemetry, and tracing. The README documents the plugin abstraction and optional ETCD coordination; sampled source confirms backend interfaces, plugin management, and selected engine behavior. Many plugin implementations were not sampled, so their connections are documented capabilities rather than inspected call paths.

Main entry points: [`src/api/cpp/backend/backend_engine.h`](https://github.com/ai-dynamo/nixl/blob/main/src/api/cpp/backend/backend_engine.h) (Backend interface), [`src/core/nixl_agent.cpp`](https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_agent.cpp) (NIXL agent), [`src/api/cpp/nixl.h`](https://github.com/ai-dynamo/nixl/blob/main/src/api/cpp/nixl.h) (C++ API), [`src/core/nixl_md_manager.cpp`](https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_md_manager.cpp) (Metadata manager)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Application APIs"]
  node_cpp_api["C++ API<br/>[nixl.h]"]
  node_python_api["Python API<br/>[_api.py]"]
  node_descriptors["Buffer descriptors<br/>[nixl_descriptors.h]"]
end

subgraph group_core["Transfer Core"]
  node_agent["NIXL agent<br/>[nixl_agent.cpp]"]
  node_plugin_manager["Plugin manager"]
  node_backend_contract["Backend interface<br/>[backend_engine.h]"]
end

subgraph group_engines["Transfer Engines"]
  node_ucx["UCX transport"]
  node_libfabric["Libfabric transport"]
  node_gds["GPU direct storage"]
  node_object_storage["Object storage<br/>[engine_impl.cpp]"]
  node_object_registry["Object engine registry"]
  node_posix["POSIX storage<br/>[posix_backend.cpp]"]
  node_azure["Azure Blob"]
end

subgraph group_coord["Coordination"]
  node_metadata_manager["Metadata manager"]
  node_p2p_metadata["Peer metadata"]
  node_etcd_metadata["ETCD metadata"]
end

subgraph group_observability["Observability"]
  node_telemetry["Telemetry core<br/>[telemetry.cpp]"]
  node_telemetry_plugins["Telemetry exporters"]
  node_tracing["Tracing core<br/>[tracer.cpp]"]
  node_nvtx["NVTX tracing"]
end

node_application(("Inference application"))
node_etcd[("ETCD service")]

node_application -.->|"calls"| node_cpp_api
node_application -.->|"calls"| node_python_api
node_cpp_api -.->|"submits requests"| node_agent
node_python_api -.->|"submits requests"| node_agent
node_cpp_api -->|"uses buffers"| node_descriptors
node_agent -.->|"loads plugins"| node_plugin_manager
node_plugin_manager -->|"creates engines"| node_backend_contract
node_backend_contract -.->|"dispatches transfers"| node_ucx
node_backend_contract -.->|"dispatches transfers"| node_libfabric
node_backend_contract -.->|"dispatches transfers"| node_gds
node_backend_contract -.->|"dispatches transfers"| node_object_storage
node_backend_contract -.->|"dispatches transfers"| node_posix
node_backend_contract -.->|"dispatches transfers"| node_azure
node_object_storage -->|"registers engine"| node_object_registry
node_agent -.->|"coordinates metadata"| node_metadata_manager
node_metadata_manager -->|"selects backend"| node_p2p_metadata
node_metadata_manager -->|"selects backend"| node_etcd_metadata
node_etcd_metadata -.->|"distributes metadata"| node_etcd
node_agent -.->|"records events"| node_telemetry
node_telemetry -.->|"exports metrics"| node_telemetry_plugins
node_agent -.->|"emits traces"| node_tracing
node_tracing -.->|"uses backend"| node_nvtx

click node_cpp_api "https://github.com/ai-dynamo/nixl/blob/main/src/api/cpp/nixl.h"
click node_python_api "https://github.com/ai-dynamo/nixl/blob/main/src/api/python/_api.py"
click node_descriptors "https://github.com/ai-dynamo/nixl/blob/main/src/api/cpp/nixl_descriptors.h"
click node_agent "https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_agent.cpp"
click node_plugin_manager "https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_plugin_manager.cpp"
click node_backend_contract "https://github.com/ai-dynamo/nixl/blob/main/src/api/cpp/backend/backend_engine.h"
click node_ucx "https://github.com/ai-dynamo/nixl/blob/main/src/plugins/ucx/ucx_thread_pool_engine.cpp"
click node_libfabric "https://github.com/ai-dynamo/nixl/blob/main/src/utils/libfabric/libfabric_rail_manager.cpp"
click node_gds "https://github.com/ai-dynamo/nixl/blob/main/src/plugins/cuda_gds/gds_batch_engine.cpp"
click node_object_storage "https://github.com/ai-dynamo/nixl/blob/main/src/plugins/obj/s3/engine_impl.cpp"
click node_object_registry "https://github.com/ai-dynamo/nixl/blob/main/src/plugins/obj/obj_engine_registry.h"
click node_posix "https://github.com/ai-dynamo/nixl/blob/main/src/plugins/posix/posix_backend.cpp"
click node_azure "https://github.com/ai-dynamo/nixl/blob/main/src/plugins/azure_blob/azure_blob_backend.cpp"
click node_metadata_manager "https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_md_manager.cpp"
click node_p2p_metadata "https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_p2p_metadata_backend.cpp"
click node_etcd_metadata "https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_etcd_metadata_backend.cpp"
click node_telemetry "https://github.com/ai-dynamo/nixl/blob/main/src/core/telemetry/telemetry.cpp"
click node_telemetry_plugins "https://github.com/ai-dynamo/nixl/blob/main/src/plugins/telemetry/prometheus/prometheus_exporter.cpp"
click node_tracing "https://github.com/ai-dynamo/nixl/blob/main/src/core/tracing/tracer.cpp"
click node_nvtx "https://github.com/ai-dynamo/nixl/blob/main/src/plugins/tracing/nvtx/nvtx_trace_backend.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cpp_api,node_python_api,node_descriptors toneBlue
class node_agent,node_plugin_manager,node_backend_contract,node_etcd toneAmber
class node_ucx,node_libfabric,node_gds,node_object_storage,node_object_registry,node_posix,node_azure toneMint
class node_metadata_manager,node_p2p_metadata,node_etcd_metadata toneRose
class node_telemetry,node_telemetry_plugins,node_tracing,node_nvtx,node_application toneIndigo
```

## Components

### Application APIs

- [C++ API](https://github.com/ai-dynamo/nixl/blob/main/src/api/cpp/nixl.h): `src/api/cpp/nixl.h`
- [Python API](https://github.com/ai-dynamo/nixl/blob/main/src/api/python/_api.py): `src/api/python/_api.py`
- [Buffer descriptors](https://github.com/ai-dynamo/nixl/blob/main/src/api/cpp/nixl_descriptors.h): `src/api/cpp/nixl_descriptors.h`

### Transfer Core

- [NIXL agent](https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_agent.cpp): `src/core/nixl_agent.cpp`
- [Plugin manager](https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_plugin_manager.cpp): `src/core/nixl_plugin_manager.cpp`
- [Backend interface](https://github.com/ai-dynamo/nixl/blob/main/src/api/cpp/backend/backend_engine.h): `src/api/cpp/backend/backend_engine.h`

### Transfer Engines

- [UCX transport](https://github.com/ai-dynamo/nixl/blob/main/src/plugins/ucx/ucx_thread_pool_engine.cpp): `src/plugins/ucx/ucx_thread_pool_engine.cpp`
- [Libfabric transport](https://github.com/ai-dynamo/nixl/blob/main/src/utils/libfabric/libfabric_rail_manager.cpp): `src/utils/libfabric/libfabric_rail_manager.cpp`
- [GPU direct storage](https://github.com/ai-dynamo/nixl/blob/main/src/plugins/cuda_gds/gds_batch_engine.cpp): `src/plugins/cuda_gds/gds_batch_engine.cpp`
- [Object storage](https://github.com/ai-dynamo/nixl/blob/main/src/plugins/obj/s3/engine_impl.cpp): `src/plugins/obj/s3/engine_impl.cpp`
- [Object engine registry](https://github.com/ai-dynamo/nixl/blob/main/src/plugins/obj/obj_engine_registry.h): `src/plugins/obj/obj_engine_registry.h`
- [POSIX storage](https://github.com/ai-dynamo/nixl/blob/main/src/plugins/posix/posix_backend.cpp): `src/plugins/posix/posix_backend.cpp`
- [Azure Blob](https://github.com/ai-dynamo/nixl/blob/main/src/plugins/azure_blob/azure_blob_backend.cpp): `src/plugins/azure_blob/azure_blob_backend.cpp`

### Coordination

- [Metadata manager](https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_md_manager.cpp): `src/core/nixl_md_manager.cpp`
- [Peer metadata](https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_p2p_metadata_backend.cpp): `src/core/nixl_p2p_metadata_backend.cpp`
- [ETCD metadata](https://github.com/ai-dynamo/nixl/blob/main/src/core/nixl_etcd_metadata_backend.cpp): `src/core/nixl_etcd_metadata_backend.cpp`

### Observability

- [Telemetry core](https://github.com/ai-dynamo/nixl/blob/main/src/core/telemetry/telemetry.cpp): `src/core/telemetry/telemetry.cpp`
- [Telemetry exporters](https://github.com/ai-dynamo/nixl/blob/main/src/plugins/telemetry/prometheus/prometheus_exporter.cpp): `src/plugins/telemetry/prometheus/prometheus_exporter.cpp`
- [Tracing core](https://github.com/ai-dynamo/nixl/blob/main/src/core/tracing/tracer.cpp): `src/core/tracing/tracer.cpp`
- [NVTX tracing](https://github.com/ai-dynamo/nixl/blob/main/src/plugins/tracing/nvtx/nvtx_trace_backend.cpp): `src/plugins/tracing/nvtx/nvtx_trace_backend.cpp`

### Other components

- **Inference application**
- **ETCD service**

## Connections

- Inference application → C++ API: calls
- Inference application → Python API: calls
- C++ API → NIXL agent: submits requests
- Python API → NIXL agent: submits requests
- C++ API → Buffer descriptors: uses buffers (evidence: `src/api/cpp/nixl.h`)
- NIXL agent → Plugin manager: loads plugins
- Plugin manager → Backend interface: creates engines (evidence: `src/core/nixl_plugin_manager.cpp`)
- Backend interface → UCX transport: dispatches transfers
- Backend interface → Libfabric transport: dispatches transfers
- Backend interface → GPU direct storage: dispatches transfers
- Backend interface → Object storage: dispatches transfers
- Backend interface → POSIX storage: dispatches transfers
- Backend interface → Azure Blob: dispatches transfers
- Object storage → Object engine registry: registers engine
- NIXL agent → Metadata manager: coordinates metadata
- Metadata manager → Peer metadata: selects backend (evidence: `src/core/nixl_md_manager.cpp`)
- Metadata manager → ETCD metadata: selects backend (evidence: `src/core/nixl_md_manager.cpp`)
- ETCD metadata → ETCD service: distributes metadata (evidence: `README.md`)
- NIXL agent → Telemetry core: records events
- Telemetry core → Telemetry exporters: exports metrics
- NIXL agent → Tracing core: emits traces
- Tracing core → NVTX tracing: uses backend
