How scalex-io/ugds works
uGDS is a user-space GPU-direct storage library: an application registers a block-device handle and GPU buffer, then issues synchronous, batch, or asynchronous transfers. The library constructs NVMe commands and manages queues; its Linux driver exposes the PCI device and maps GPU memory for DMA, allowing the SSD to transfer directly to or from GPU memory. The README also documents CUDA/HIP backends, DMA-buf export, RDMA support, hugepages, and interrupt completion. Edges reflect README statements or references in sampled source/index; unsampled implementation details are not inferred.
Main entry points: pci.c PCI Driver · ugds_io.cpp Sync I/O · linux_dma.cpp DMA Mapping · ugds_internal.h uGDS State
Components 26
uGDS API
- Sync I/O
src/ugds_io.cpp - Batch I/O
src/ugds_batch.cpp - Async I/O
src/ugds_async.cpp - Handle Lifecycle
src/ugds_handle.cpp - uGDS State
src/ugds_internal.h
NVMe Library
- NVMe Admin
src/libnvm/admin.cpp - Controller Setup
src/libnvm/ctrl.cpp - Queue Management
src/libnvm/queue.cpp - DMA Mapping
src/libnvm/linux_dma.cpp - Linux Device
src/libnvm/linux_device.cpp - RPC
src/libnvm/rpc.cpp - DMA Descriptors
src/libnvm/internal/dma.h - Controller State
src/libnvm/internal/lib_ctrl.h - NVMe Registers
src/libnvm/internal/regs.h
Kernel Driver
- PCI Driver
drv/pci.c - GPU Page Mapping
drv/map.c - Driver Controller
drv/ctrl.h - Interrupt Handling
drv/irq.h - Driver Lists
drv/list.h
GPU Memory
- Buffer Registration
src/ugds_buf.cpp - RDMA Registration
src/ugds_rdma.cpp - Hugepages
src/ugds_hugepage.cpp
Other components
- Application
- NVMe SSD
- GPU Memory
- RDMA Consumer
Connections 25
- Application → Handle Lifecycle: registers handle
- Application → Buffer Registration: registers buffer
- Application → Sync I/O: issues transfer
- Application → Batch I/O: submits batch
- Application → Async I/O: issues async
- Sync I/O → NVMe SSD: transfers data
- Sync I/O → GPU Memory: transfers data
- Application → RDMA Registration: registers memory
- RDMA Consumer → GPU Memory: uses exported memory
- Buffer Registration → DMA Mapping: maps GPU buffer
src/ugds_buf.cpp - DMA Mapping → DMA Descriptors: initializes mapping
src/libnvm/linux_dma.cpp - DMA Mapping → Controller State: uses controller
src/libnvm/linux_dma.cpp - Controller Setup → Controller State: uses state
src/libnvm/ctrl.cpp - NVMe Admin → NVMe Registers: accesses registers
src/libnvm/admin.cpp - NVMe Admin → RPC: uses RPC
src/libnvm/admin.cpp - RPC → Controller State: uses controller
src/libnvm/rpc.cpp - PCI Driver → Driver Controller: manages controller
drv/pci.c - PCI Driver → GPU Page Mapping: maps GPU pages
drv/pci.c - PCI Driver → Interrupt Handling: registers interrupts
drv/pci.c - GPU Page Mapping → Driver Lists: walks controller list
drv/map.c - Driver Controller → Driver Lists: stores list node
drv/ctrl.h - Async I/O → uGDS State: uses shared state
src/ugds_async.cpp - Batch I/O → uGDS State: uses shared state
src/ugds_batch.cpp - Sync I/O → uGDS State: uses shared state
src/ugds_io.cpp - PCI Driver → NVMe SSD: exposes PCI device