How danielesavino/nccl works
NCCL is a GPU communication library for coordinating collectives and point-to-point traffic across GPUs, using local interconnects or network transports. The principal workflow is an application submitting communication work through NCCL, which is planned for GPU execution and carried by an appropriate transport. The graph also preserves topology discovery, proxy progress, memory management, plugins, and the Python binding. Relationships are limited to README-documented behavior and source-index references; unsampled implementation details are not assumed.
Main entry points: init.cc NCCL API · comm.h Communicator state · communicator.py Python communicator · net.cc Network transport
Components 22
API and coordination
- NCCL API
src/init.cc - Communicator state
src/include/comm.h - Work enqueue
src/enqueue.cc - Memory manager
src/mem_manager.cc - Tracked GPU memory
src/include/mem_manager.h
GPU execution
- Collective operations
src/collectives.cc - GPU primitives
src/device/prims_simple.h
Topology and transport
- Topology search
src/graph/search.cc - Network transport
src/transport/net.cc - Proxy progress
src/proxy.cc - Rank bootstrap
src/bootstrap.cc
Plugins and services
- Profiler plugin
src/plugin/profiler.cc - Profiler inspector
plugins/profiler/inspector/inspector.cc - Tuner plugin
src/plugin/tuner.cc - Network plugin API
src/plugin/net.cc - RAS monitoring
src/ras/ras.cc
Language bindings
- Python communicator
bindings/nccl4py/nccl/core/communicator.py - Python buffers
bindings/nccl4py/nccl/core/buffer.py - Python resources
bindings/nccl4py/nccl/core/resources.py
Other components
- GPU application
- CUDA GPUs
- InfiniBand or TCP
Connections 18
- GPU application → NCCL API: calls communication
- NCCL API → Collective operations: offers collectives
- Collective operations → GPU primitives: executes primitives
- GPU application → NCCL API: uses send and receive
- NCCL API → Communicator state: uses communicator state
- Communicator state → Work enqueue: queues work
- Topology search → Communicator state: reads communicator
src/graph/search.cc - Topology search → Network transport: references transport
src/graph/search.cc - Network transport → InfiniBand or TCP: communicates over
- Network transport → Proxy progress: uses proxy services
- Rank bootstrap → InfiniBand or TCP: coordinates ranks
- Memory manager → Tracked GPU memory: tracks allocations
src/mem_manager.cc - Memory manager → Communicator state: manages per-communicator memory
src/mem_manager.cc - Profiler plugin → Profiler inspector: provides profiling data
- Python communicator → Python buffers: uses buffers
bindings/nccl4py/nccl/core/communicator.py - Python communicator → Python resources: uses resources
bindings/nccl4py/nccl/core/communicator.py - Python communicator → NCCL API: wraps NCCL
- GPU primitives → CUDA GPUs: runs on