# axio-project/fuselink architecture

> Architecture diagram of the GitHub repository axio-project/fuselink, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/axio-project/fuselink
- Repository: https://github.com/axio-project/fuselink

## Overview

1. **What this repository is**
   - FuseLink is a **NCCL network plugin** that replaces/extends NCCL’s transport layer to do **efficient GPU communication over multiple RDMA-capable NICs**.
   - The main architectural boundary is: **application / training code → NCCL → FuseLink plugin → RDMA/GPU Direct RDMA → NICs and GPUs**.

2. **Core runtime component**
   - The production integration lives in `src/plugin.cc`, which is the NCCL-facing entry point.
   - This is the layer that exposes FuseLink as `libnccl-net-fuselink.so` and makes it selectable through `NCCL_NET_PLUGIN=fuselink`.

3. **Memory and data movement layer**
   - `src/cumem.cu` and `src/cumem.h` implement CUDA-side memory operations used by the transport.
   - This is where GPU memory interaction, peer access, and GPU Direct RDMA-adjacent behavior are centralized, so it sits directly on the data path.

4. **FuseLink internal state and transport structures**
   - `src/fuselink.cc` and `src/fuselink.h` hold the plugin’s internal memory/connection structures.
   - This is the repository’s main control/data model for tracking how traffic is distributed across NICs and how communication resources are organized.

5. **RDMA and networking wrappers**
   - `src/extern/ibvcore.h`, `ibvwrap.h`, and `nccl_net.h` provide the low-level verbs/NCCL interfaces the plugin builds on.
   - `src/extern/socket.cc/.h` and `utils.h` are support utilities, likely used for control-plane or fallback communication rather than the fast GPU data path.
   - `param.h` and `timer.h` are auxiliary support headers borrowed from NCCL.

6. **Monitoring subsystem**
   - `src/monitor.cpp`, `src/monitor.h`, and `src/monitor_main.cpp` implement a separate monitor process/component.
   - The README calls this out as monitoring “NIC idleness,” so architecturally this is a side subsystem that observes NIC utilization and likely feeds scheduling/selection decisions in the plugin or related tooling.

7. **Build and packaging boundaries**
   - The repo is split between building **NCCL itself** and building **the FuseLink plugin** against that NCCL tree.
   - The documented build flow uses `make src.build` for NCCL and then `make fl` for the plugin, with `NCCL_BUILD_DIR` and `CUDA_HOME` controlling the linkage/runtime environment.

8. **Deployment/runtime requirements**
   - FuseLink depends on **RDMA-capable NICs**, **GPU Direct RDMA**, and preferably **NVLink** for intra-node GPU communication efficiency.
   - It also assumes specific software versions: NCCL 2.19.4, libibverbs, CUDA ≥ 11.8, and an NVIDIA driver version tested by the authors.
   - The `scripts/disable_asc.sh` helper is part of the deployment boundary because PCIe ACS must be disabled for fast NIC access.

9. **Serving/benchmark application layer**
   - `experiments/serving` is a substantial application harness for testing the plugin in model-serving scenarios.
   - It includes CMake build files, scheduler logic, model wrappers, online/offline entrypoints, and batch scripts for benchmark execution.
   - This directory is where the plugin is exercised under realistic multi-GPU, multi-NIC serving loads.

10. **Model-serving data flow**
   - `experiments/serving/main.cc` and `online_main.cc` are the main executables for serving-style experiments.
   - `model.hh` and `online_model.hh` define the model-side abstraction, while `scheduler.h` suggests request/task placement or concurrency control around communication-heavy workloads.
   - The included `SwiftTransformer` submodule indicates the repository can integrate with large-model serving code paths.

11. **Experimentation and evaluation harness**
   - `experiments/bw_test`, `experiments/dlrm`, and `experiments/ep` provide benchmark or research-specific testbeds.
   - `experiments/serving/experiment/common/gen_input.py` generates inputs for repeatable experiments, and the `.sbatch` files indicate the repo is designed to run in scheduled HPC/cluster environments.
   - These are not the core runtime, but they define how the communication layer is validated under different workload shapes.

12. **External code and submodules**
   - `nccl/` is a checked-in NCCL dependency used for building and plugin integration.
   - `experiments/ep/Megatron-DeepSpeed` and `experiments/serving/SwiftTransformer` are external model/framework submodules that serve as workload generators or integration targets.

13. **How to draw the architecture**
   - A clear diagram should show three main planes:
     - **Application plane:** training/serving/benchmark apps in `experiments/*`
     - **Communication plane:** NCCL + FuseLink plugin + monitor
     - **Hardware plane:** GPUs, NICs, RDMA fabric, NVLink/PCIe
   - The primary flow is application collective communication requests entering NCCL, being intercepted by FuseLink, then routed over multiple NICs with GPU-aware memory handling and monitoring support.

Main entry points: [`src/fuselink.cc`](https://github.com/axio-project/fuselink/blob/main/src/fuselink.cc) (FuseLink core), [`src/plugin.cc`](https://github.com/axio-project/fuselink/blob/main/src/plugin.cc) (Plugin entry), [`nccl`](https://github.com/axio-project/fuselink/tree/main/nccl) (NCCL tree), [`src/cumem.cu`](https://github.com/axio-project/fuselink/blob/main/src/cumem.cu) (GPU memory)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_application["Application plane"]
  node_experiments_serving["Serving harness<br/>serving workloads"]
  node_experiments_bw_test["BW test<br/>bandwidth benchmark"]
  node_experiments_dlrm["DLRM<br/>benchmark workload"]
  node_experiments_ep["EP suite<br/>distributed training workloads"]
  node_swift_transformer["SwiftTransformer<br/>model submodule"]
  node_megatron_ds["Megatron-DeepSpeed<br/>model submodule"]
end

subgraph group_communication["Communication plane"]
  node_plugin_entry["Plugin entry<br/>NCCL net plugin<br/>[plugin.cc]"]
  node_fuselink_core["FuseLink core<br/>transport state<br/>[fuselink.cc]"]
  node_gpu_mem["GPU memory<br/>CUDA transport memory<br/>[cumem.cu]"]
  node_monitor["Monitor<br/>NIC observer<br/>[monitor_main.cpp]"]
end

subgraph group_runtime["Runtime and external deps"]
  node_nccl_tree["NCCL tree<br/>upstream NCCL"]
  node_verbs_wrap["Verbs wrappers<br/>RDMA interface<br/>[ibvwrap.h]"]
  node_ibv_core["IBV core<br/>verbs core<br/>[ibvcore.h]"]
  node_socket_support["Socket support<br/>control-plane network<br/>[socket.cc]"]
  node_deploy_script["ACS disable<br/>deployment script<br/>[disable_asc.sh]"]
end

subgraph group_hardware["Hardware plane"]
  node_rdma_fabric{{"RDMA fabric<br/>network fabric"}}
  node_gpus{{"GPUs<br/>accelerators"}}
  node_nics{{"NICs<br/>rdma-capable devices"}}
  node_nvlink_pcie{{"NVLink/PCIe<br/>intra-node interconnect"}}
end

node_experiments_serving -->|"uses NCCL"| node_nccl_tree
node_experiments_bw_test -->|"uses NCCL"| node_nccl_tree
node_experiments_dlrm -->|"uses NCCL"| node_nccl_tree
node_experiments_ep -->|"uses NCCL"| node_nccl_tree
node_experiments_serving -->|"integrates"| node_swift_transformer
node_experiments_ep -->|"integrates"| node_megatron_ds
node_nccl_tree -->|"loads plugin"| node_plugin_entry
node_plugin_entry -->|"delegates"| node_fuselink_core
node_fuselink_core -->|"uses"| node_gpu_mem
node_fuselink_core -->|"uses"| node_verbs_wrap
node_verbs_wrap -->|"wraps"| node_ibv_core
node_plugin_entry -.->|"control path"| node_socket_support
node_monitor -.->|"observes"| node_fuselink_core
node_monitor -.->|"measures"| node_nics
node_fuselink_core -->|"spreads traffic"| node_nics
node_fuselink_core -->|"sends over"| node_rdma_fabric
node_gpu_mem -->|"accesses"| node_gpus
node_gpu_mem -->|"depends on"| node_nvlink_pcie
node_deploy_script -.->|"configures"| node_nvlink_pcie
node_plugin_entry -->|"serves"| node_gpus
node_plugin_entry -->|"serves"| node_nics

click node_experiments_serving "https://github.com/axio-project/fuselink/tree/main/experiments/serving"
click node_experiments_bw_test "https://github.com/axio-project/fuselink/tree/main/experiments/bw_test"
click node_experiments_dlrm "https://github.com/axio-project/fuselink/tree/main/experiments/dlrm"
click node_experiments_ep "https://github.com/axio-project/fuselink/tree/main/experiments/ep"
click node_swift_transformer "https://github.com/axio-project/fuselink/tree/main/experiments/serving/SwiftTransformer"
click node_megatron_ds "https://github.com/axio-project/fuselink/tree/main/experiments/ep/Megatron-DeepSpeed"
click node_nccl_tree "https://github.com/axio-project/fuselink/tree/main/nccl"
click node_plugin_entry "https://github.com/axio-project/fuselink/blob/main/src/plugin.cc"
click node_fuselink_core "https://github.com/axio-project/fuselink/blob/main/src/fuselink.cc"
click node_gpu_mem "https://github.com/axio-project/fuselink/blob/main/src/cumem.cu"
click node_monitor "https://github.com/axio-project/fuselink/blob/main/src/monitor_main.cpp"
click node_verbs_wrap "https://github.com/axio-project/fuselink/blob/main/src/extern/ibvwrap.h"
click node_ibv_core "https://github.com/axio-project/fuselink/blob/main/src/extern/ibvcore.h"
click node_socket_support "https://github.com/axio-project/fuselink/blob/main/src/extern/socket.cc"
click node_deploy_script "https://github.com/axio-project/fuselink/blob/main/scripts/disable_asc.sh"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_experiments_serving,node_experiments_bw_test,node_experiments_dlrm,node_experiments_ep,node_swift_transformer,node_megatron_ds toneBlue
class node_plugin_entry,node_fuselink_core,node_gpu_mem,node_monitor toneAmber
class node_nccl_tree,node_verbs_wrap,node_ibv_core,node_socket_support,node_deploy_script toneMint
class node_rdma_fabric,node_gpus,node_nics,node_nvlink_pcie toneRose
```

## Components

### Application plane

Serving and benchmark workloads that drive NCCL collectives through FuseLink.

- [Serving harness](https://github.com/axio-project/fuselink/tree/main/experiments/serving): serving workloads · Main model-serving benchmark and execution harness. · `experiments/serving`
- [BW test](https://github.com/axio-project/fuselink/tree/main/experiments/bw_test): bandwidth benchmark · Bandwidth-focused experiment entrypoint. · `experiments/bw_test`
- [DLRM](https://github.com/axio-project/fuselink/tree/main/experiments/dlrm): benchmark workload · Recommendation-model experiment target. · `experiments/dlrm`
- [EP suite](https://github.com/axio-project/fuselink/tree/main/experiments/ep): distributed training workloads · Experiment area for large-scale model-parallel training. · `experiments/ep`
- [SwiftTransformer](https://github.com/axio-project/fuselink/tree/main/experiments/serving/SwiftTransformer): model submodule · External serving/model integration used by the harness. · `experiments/serving/SwiftTransformer`
- [Megatron-DeepSpeed](https://github.com/axio-project/fuselink/tree/main/experiments/ep/Megatron-DeepSpeed): model submodule · External training stack used in EP experiments. · `experiments/ep/Megatron-DeepSpeed`

### Communication plane

NCCL integration, transport logic, memory movement, and monitoring.

- [Plugin entry](https://github.com/axio-project/fuselink/blob/main/src/plugin.cc): NCCL net plugin · NCCL-facing entry point exposed as libnccl-net-fuselink.so. · `src/plugin.cc`
- [FuseLink core](https://github.com/axio-project/fuselink/blob/main/src/fuselink.cc): transport state · Internal connection, resource, and traffic-distribution model. · `src/fuselink.cc`
- [GPU memory](https://github.com/axio-project/fuselink/blob/main/src/cumem.cu): CUDA transport memory · CUDA-side memory operations and GPU Direct RDMA-adjacent behavior. · `src/cumem.cu`
- [Monitor](https://github.com/axio-project/fuselink/blob/main/src/monitor_main.cpp): NIC observer · Separate process that watches NIC idleness and related signals. · `src/monitor_main.cpp`

### Runtime and external deps

Checked-in NCCL tree and low-level verbs/socket support used by the plugin.

- [NCCL tree](https://github.com/axio-project/fuselink/tree/main/nccl): upstream NCCL · Checked-in NCCL dependency and build target for plugin integration. · `nccl`
- [Verbs wrappers](https://github.com/axio-project/fuselink/blob/main/src/extern/ibvwrap.h): RDMA interface · Low-level ibverbs and NCCL network interface wrappers. · `src/extern/ibvwrap.h`
- [IBV core](https://github.com/axio-project/fuselink/blob/main/src/extern/ibvcore.h): verbs core · Core verbs helpers and RDMA plumbing used by the plugin. · `src/extern/ibvcore.h`
- [Socket support](https://github.com/axio-project/fuselink/blob/main/src/extern/socket.cc): control-plane network · Auxiliary socket utilities for setup or fallback communication. · `src/extern/socket.cc`
- [ACS disable](https://github.com/axio-project/fuselink/blob/main/scripts/disable_asc.sh): deployment script · Helper for disabling PCIe ACS to improve device access. · `scripts/disable_asc.sh`

### Hardware plane

GPU, NIC, and interconnect resources targeted by the plugin.

- **RDMA fabric**: network fabric · Multi-NIC RDMA path used for transport traffic.
- **GPUs**: accelerators · GPU devices providing collective communication endpoints and memory.
- **NICs**: rdma-capable devices · Multiple network interfaces used for load distribution.
- **NVLink/PCIe**: intra-node interconnect · Local GPU interconnect and PCIe access constraints.

## Connections

- Serving harness → NCCL tree: uses NCCL · Serving workloads rely on NCCL collectives.
- BW test → NCCL tree: uses NCCL · Benchmark harness drives the communication stack.
- DLRM → NCCL tree: uses NCCL · Training benchmark exercises NCCL transports.
- EP suite → NCCL tree: uses NCCL · Model-parallel experiments depend on NCCL.
- Serving harness → SwiftTransformer: integrates · Serving harness wraps the transformer workload.
- EP suite → Megatron-DeepSpeed: integrates · EP experiments use Megatron-DeepSpeed as workload source.
- NCCL tree → Plugin entry: loads plugin · NCCL resolves the FuseLink network plugin.
- Plugin entry → FuseLink core: delegates · Entry point hands transport setup and operations to core state.
- FuseLink core → GPU memory: uses · Core transport relies on CUDA-side memory movement.
- FuseLink core → Verbs wrappers: uses · Core transport builds on RDMA verbs abstractions.
- Verbs wrappers → IBV core: wraps · Wrapper layer sits on lower-level verbs helpers.
- Plugin entry → Socket support: control path · Plugin may use socket utilities for auxiliary communication.
- Monitor → FuseLink core: observes · Monitor tracks NIC idleness relevant to plugin decisions.
- Monitor → NICs: measures · Side process watches NIC utilization.
- FuseLink core → NICs: spreads traffic · Transport distributes communication across multiple NICs.
- FuseLink core → RDMA fabric: sends over · Primary data path rides the RDMA fabric.
- GPU memory → GPUs: accesses · Memory layer operates on GPU-resident buffers.
- GPU memory → NVLink/PCIe: depends on · Efficient local transfers depend on GPU interconnect and PCIe behavior.
- ACS disable → NVLink/PCIe: configures · Deployment helper disables ACS to improve PCIe access paths.
- Plugin entry → GPUs: serves · Plugin exposes GPU-aware transport to the application stack.
- Plugin entry → NICs: serves · Plugin routes communication through RDMA-capable NICs.
