# danielesavino/nccl architecture

> Architecture diagram of the GitHub repository danielesavino/nccl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/danielesavino/nccl
- Repository: https://github.com/danielesavino/nccl

## Overview

NCCL is a GPU communication library for coordinating collectives and point-to-point traffic across GPUs, using local interconnects or network transports. The principal workflow is an application submitting communication work through NCCL, which is planned for GPU execution and carried by an appropriate transport. The graph also preserves topology discovery, proxy progress, memory management, plugins, and the Python binding. Relationships are limited to README-documented behavior and source-index references; unsampled implementation details are not assumed.

Main entry points: [`src/init.cc`](https://github.com/danielesavino/nccl/blob/master/src/init.cc) (NCCL API), [`src/include/comm.h`](https://github.com/danielesavino/nccl/blob/master/src/include/comm.h) (Communicator state), [`bindings/nccl4py/nccl/core/communicator.py`](https://github.com/danielesavino/nccl/blob/master/bindings/nccl4py/nccl/core/communicator.py) (Python communicator), [`src/transport/net.cc`](https://github.com/danielesavino/nccl/blob/master/src/transport/net.cc) (Network transport)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and coordination"]
  node_api["NCCL API<br/>[init.cc]"]
  node_communicator["Communicator state<br/>[comm.h]"]
  node_enqueue["Work enqueue<br/>[enqueue.cc]"]
  node_memory["Memory manager<br/>[mem_manager.cc]"]
  node_memory_state[("Tracked GPU memory<br/>[mem_manager.h]")]
end

subgraph group_execution["GPU execution"]
  node_collectives["Collective operations<br/>[collectives.cc]"]
  node_device["GPU primitives<br/>[prims_simple.h]"]
end

subgraph group_connectivity["Topology and transport"]
  node_topology["Topology search<br/>[search.cc]"]
  node_transport["Network transport<br/>[net.cc]"]
  node_proxy["Proxy progress<br/>[proxy.cc]"]
  node_bootstrap["Rank bootstrap<br/>[bootstrap.cc]"]
end

subgraph group_extensions["Plugins and services"]
  node_profiler["Profiler plugin<br/>[profiler.cc]"]
  node_inspector["Profiler inspector<br/>[inspector.cc]"]
  node_tuner["Tuner plugin<br/>[tuner.cc]"]
  node_net_plugin["Network plugin API<br/>[net.cc]"]
  node_ras["RAS monitoring<br/>[ras.cc]"]
end

subgraph group_bindings["Language bindings"]
  node_python["Python communicator<br/>[communicator.py]"]
  node_python_buffer["Python buffers<br/>[buffer.py]"]
  node_python_resources["Python resources<br/>[resources.py]"]
end

node_application(("GPU application"))
node_gpu(("CUDA GPUs"))
node_network(("InfiniBand or TCP"))

node_application -->|"calls communication"| node_api
node_api -->|"offers collectives"| node_collectives
node_collectives -.->|"executes primitives"| node_device
node_application -->|"uses send and receive"| node_api
node_api -.->|"uses communicator state"| node_communicator
node_communicator -.->|"queues work"| node_enqueue
node_topology -->|"reads communicator"| node_communicator
node_topology -->|"references transport"| node_transport
node_transport -->|"communicates over"| node_network
node_transport -.->|"uses proxy services"| node_proxy
node_bootstrap -.->|"coordinates ranks"| node_network
node_memory -->|"tracks allocations"| node_memory_state
node_memory -->|"manages per-communicator memory"| node_communicator
node_profiler -.->|"provides profiling data"| node_inspector
node_python -->|"uses buffers"| node_python_buffer
node_python -->|"uses resources"| node_python_resources
node_python -.->|"wraps NCCL"| node_api
node_device -->|"runs on"| node_gpu

click node_api "https://github.com/danielesavino/nccl/blob/master/src/init.cc"
click node_communicator "https://github.com/danielesavino/nccl/blob/master/src/include/comm.h"
click node_enqueue "https://github.com/danielesavino/nccl/blob/master/src/enqueue.cc"
click node_collectives "https://github.com/danielesavino/nccl/blob/master/src/collectives.cc"
click node_device "https://github.com/danielesavino/nccl/blob/master/src/device/prims_simple.h"
click node_topology "https://github.com/danielesavino/nccl/blob/master/src/graph/search.cc"
click node_transport "https://github.com/danielesavino/nccl/blob/master/src/transport/net.cc"
click node_proxy "https://github.com/danielesavino/nccl/blob/master/src/proxy.cc"
click node_bootstrap "https://github.com/danielesavino/nccl/blob/master/src/bootstrap.cc"
click node_memory "https://github.com/danielesavino/nccl/blob/master/src/mem_manager.cc"
click node_memory_state "https://github.com/danielesavino/nccl/blob/master/src/include/mem_manager.h"
click node_profiler "https://github.com/danielesavino/nccl/blob/master/src/plugin/profiler.cc"
click node_inspector "https://github.com/danielesavino/nccl/blob/master/plugins/profiler/inspector/inspector.cc"
click node_tuner "https://github.com/danielesavino/nccl/blob/master/src/plugin/tuner.cc"
click node_net_plugin "https://github.com/danielesavino/nccl/blob/master/src/plugin/net.cc"
click node_ras "https://github.com/danielesavino/nccl/blob/master/src/ras/ras.cc"
click node_python "https://github.com/danielesavino/nccl/blob/master/bindings/nccl4py/nccl/core/communicator.py"
click node_python_buffer "https://github.com/danielesavino/nccl/blob/master/bindings/nccl4py/nccl/core/buffer.py"
click node_python_resources "https://github.com/danielesavino/nccl/blob/master/bindings/nccl4py/nccl/core/resources.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_communicator,node_enqueue,node_memory,node_memory_state toneBlue
class node_collectives,node_device toneAmber
class node_topology,node_transport,node_proxy,node_bootstrap toneMint
class node_profiler,node_inspector,node_tuner,node_net_plugin,node_ras toneRose
class node_python,node_python_buffer,node_python_resources,node_application,node_gpu,node_network toneIndigo
```

## Components

### API and coordination

- [NCCL API](https://github.com/danielesavino/nccl/blob/master/src/init.cc): `src/init.cc`
- [Communicator state](https://github.com/danielesavino/nccl/blob/master/src/include/comm.h): `src/include/comm.h`
- [Work enqueue](https://github.com/danielesavino/nccl/blob/master/src/enqueue.cc): `src/enqueue.cc`
- [Memory manager](https://github.com/danielesavino/nccl/blob/master/src/mem_manager.cc): `src/mem_manager.cc`
- [Tracked GPU memory](https://github.com/danielesavino/nccl/blob/master/src/include/mem_manager.h): `src/include/mem_manager.h`

### GPU execution

- [Collective operations](https://github.com/danielesavino/nccl/blob/master/src/collectives.cc): `src/collectives.cc`
- [GPU primitives](https://github.com/danielesavino/nccl/blob/master/src/device/prims_simple.h): `src/device/prims_simple.h`

### Topology and transport

- [Topology search](https://github.com/danielesavino/nccl/blob/master/src/graph/search.cc): `src/graph/search.cc`
- [Network transport](https://github.com/danielesavino/nccl/blob/master/src/transport/net.cc): `src/transport/net.cc`
- [Proxy progress](https://github.com/danielesavino/nccl/blob/master/src/proxy.cc): `src/proxy.cc`
- [Rank bootstrap](https://github.com/danielesavino/nccl/blob/master/src/bootstrap.cc): `src/bootstrap.cc`

### Plugins and services

- [Profiler plugin](https://github.com/danielesavino/nccl/blob/master/src/plugin/profiler.cc): `src/plugin/profiler.cc`
- [Profiler inspector](https://github.com/danielesavino/nccl/blob/master/plugins/profiler/inspector/inspector.cc): `plugins/profiler/inspector/inspector.cc`
- [Tuner plugin](https://github.com/danielesavino/nccl/blob/master/src/plugin/tuner.cc): `src/plugin/tuner.cc`
- [Network plugin API](https://github.com/danielesavino/nccl/blob/master/src/plugin/net.cc): `src/plugin/net.cc`
- [RAS monitoring](https://github.com/danielesavino/nccl/blob/master/src/ras/ras.cc): `src/ras/ras.cc`

### Language bindings

- [Python communicator](https://github.com/danielesavino/nccl/blob/master/bindings/nccl4py/nccl/core/communicator.py): `bindings/nccl4py/nccl/core/communicator.py`
- [Python buffers](https://github.com/danielesavino/nccl/blob/master/bindings/nccl4py/nccl/core/buffer.py): `bindings/nccl4py/nccl/core/buffer.py`
- [Python resources](https://github.com/danielesavino/nccl/blob/master/bindings/nccl4py/nccl/core/resources.py): `bindings/nccl4py/nccl/core/resources.py`

### Other components

- **GPU application**
- **CUDA GPUs**
- **InfiniBand or TCP**

## Connections

- GPU application → NCCL API: calls communication
- NCCL API → Collective operations: offers collectives
- Collective operations → GPU primitives: executes primitives
- GPU application → NCCL API: uses send and receive
- NCCL API → Communicator state: uses communicator state
- Communicator state → Work enqueue: queues work
- Topology search → Communicator state: reads communicator (evidence: `src/graph/search.cc`)
- Topology search → Network transport: references transport (evidence: `src/graph/search.cc`)
- Network transport → InfiniBand or TCP: communicates over
- Network transport → Proxy progress: uses proxy services
- Rank bootstrap → InfiniBand or TCP: coordinates ranks
- Memory manager → Tracked GPU memory: tracks allocations (evidence: `src/mem_manager.cc`)
- Memory manager → Communicator state: manages per-communicator memory (evidence: `src/mem_manager.cc`)
- Profiler plugin → Profiler inspector: provides profiling data
- Python communicator → Python buffers: uses buffers (evidence: `bindings/nccl4py/nccl/core/communicator.py`)
- Python communicator → Python resources: uses resources (evidence: `bindings/nccl4py/nccl/core/communicator.py`)
- Python communicator → NCCL API: wraps NCCL
- GPU primitives → CUDA GPUs: runs on