# scalex-io/ugds architecture

> Architecture diagram of the GitHub repository scalex-io/ugds, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/scalex-io/ugds
- Repository: https://github.com/scalex-io/ugds

## Overview

uGDS is a user-space GPU-direct storage library: an application registers a block-device handle and GPU buffer, then issues synchronous, batch, or asynchronous transfers. The library constructs NVMe commands and manages queues; its Linux driver exposes the PCI device and maps GPU memory for DMA, allowing the SSD to transfer directly to or from GPU memory. The README also documents CUDA/HIP backends, DMA-buf export, RDMA support, hugepages, and interrupt completion. Edges reflect README statements or references in sampled source/index; unsampled implementation details are not inferred.

Main entry points: [`drv/pci.c`](https://github.com/scalex-io/ugds/blob/main/drv/pci.c) (PCI Driver), [`src/ugds_io.cpp`](https://github.com/scalex-io/ugds/blob/main/src/ugds_io.cpp) (Sync I/O), [`src/libnvm/linux_dma.cpp`](https://github.com/scalex-io/ugds/blob/main/src/libnvm/linux_dma.cpp) (DMA Mapping), [`src/ugds_internal.h`](https://github.com/scalex-io/ugds/blob/main/src/ugds_internal.h) (uGDS State)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["uGDS API"]
  node_sync["Sync I/O<br/>[ugds_io.cpp]"]
  node_batch["Batch I/O<br/>[ugds_batch.cpp]"]
  node_async["Async I/O<br/>[ugds_async.cpp]"]
  node_handle["Handle Lifecycle<br/>[ugds_handle.cpp]"]
  node_ugdsstate["uGDS State<br/>[ugds_internal.h]"]
end

subgraph group_nvm["NVMe Library"]
  node_admin["NVMe Admin<br/>[admin.cpp]"]
  node_controller["Controller Setup<br/>[ctrl.cpp]"]
  node_queue["Queue Management<br/>[queue.cpp]"]
  node_nvm_dma["DMA Mapping<br/>[linux_dma.cpp]"]
  node_device["Linux Device<br/>[linux_device.cpp]"]
  node_rpc["RPC<br/>[rpc.cpp]"]
  node_dma_internal["DMA Descriptors<br/>[dma.h]"]
  node_libctrl["Controller State<br/>[lib_ctrl.h]"]
  node_regs["NVMe Registers<br/>[regs.h]"]
end

subgraph group_driver["Kernel Driver"]
  node_pci["PCI Driver<br/>[pci.c]"]
  node_map["GPU Page Mapping<br/>[map.c]"]
  node_drvctrl["Driver Controller<br/>[ctrl.h]"]
  node_irq["Interrupt Handling<br/>[irq.h]"]
  node_drvlist["Driver Lists<br/>[list.h]"]
end

subgraph group_memory["GPU Memory"]
  node_buffer["Buffer Registration<br/>[ugds_buf.cpp]"]
  node_rdma["RDMA Registration<br/>[ugds_rdma.cpp]"]
  node_hugepage["Hugepages<br/>[ugds_hugepage.cpp]"]
end

node_application(("Application"))
node_ssd[("NVMe SSD")]
node_gpu[("GPU Memory")]
node_rdma_consumer(("RDMA Consumer"))

node_application -->|"registers handle"| node_handle
node_application -->|"registers buffer"| node_buffer
node_application -->|"issues transfer"| node_sync
node_application -->|"submits batch"| node_batch
node_application -->|"issues async"| node_async
node_sync -->|"transfers data"| node_ssd
node_sync -->|"transfers data"| node_gpu
node_application -->|"registers memory"| node_rdma
node_rdma_consumer -.->|"uses exported memory"| node_gpu
node_buffer -->|"maps GPU buffer"| node_nvm_dma
node_nvm_dma -->|"initializes mapping"| node_dma_internal
node_nvm_dma -->|"uses controller"| node_libctrl
node_controller -->|"uses state"| node_libctrl
node_admin -->|"accesses registers"| node_regs
node_admin -->|"uses RPC"| node_rpc
node_rpc -->|"uses controller"| node_libctrl
node_pci -->|"manages controller"| node_drvctrl
node_pci -->|"maps GPU pages"| node_map
node_pci -->|"registers interrupts"| node_irq
node_map -->|"walks controller list"| node_drvlist
node_drvctrl -->|"stores list node"| node_drvlist
node_async -->|"uses shared state"| node_ugdsstate
node_batch -->|"uses shared state"| node_ugdsstate
node_sync -->|"uses shared state"| node_ugdsstate
node_pci -->|"exposes PCI device"| node_ssd

click node_sync "https://github.com/scalex-io/ugds/blob/main/src/ugds_io.cpp"
click node_batch "https://github.com/scalex-io/ugds/blob/main/src/ugds_batch.cpp"
click node_async "https://github.com/scalex-io/ugds/blob/main/src/ugds_async.cpp"
click node_handle "https://github.com/scalex-io/ugds/blob/main/src/ugds_handle.cpp"
click node_buffer "https://github.com/scalex-io/ugds/blob/main/src/ugds_buf.cpp"
click node_rdma "https://github.com/scalex-io/ugds/blob/main/src/ugds_rdma.cpp"
click node_hugepage "https://github.com/scalex-io/ugds/blob/main/src/ugds_hugepage.cpp"
click node_ugdsstate "https://github.com/scalex-io/ugds/blob/main/src/ugds_internal.h"
click node_admin "https://github.com/scalex-io/ugds/blob/main/src/libnvm/admin.cpp"
click node_controller "https://github.com/scalex-io/ugds/blob/main/src/libnvm/ctrl.cpp"
click node_queue "https://github.com/scalex-io/ugds/blob/main/src/libnvm/queue.cpp"
click node_nvm_dma "https://github.com/scalex-io/ugds/blob/main/src/libnvm/linux_dma.cpp"
click node_device "https://github.com/scalex-io/ugds/blob/main/src/libnvm/linux_device.cpp"
click node_rpc "https://github.com/scalex-io/ugds/blob/main/src/libnvm/rpc.cpp"
click node_dma_internal "https://github.com/scalex-io/ugds/blob/main/src/libnvm/internal/dma.h"
click node_libctrl "https://github.com/scalex-io/ugds/blob/main/src/libnvm/internal/lib_ctrl.h"
click node_regs "https://github.com/scalex-io/ugds/blob/main/src/libnvm/internal/regs.h"
click node_pci "https://github.com/scalex-io/ugds/blob/main/drv/pci.c"
click node_map "https://github.com/scalex-io/ugds/blob/main/drv/map.c"
click node_drvctrl "https://github.com/scalex-io/ugds/blob/main/drv/ctrl.h"
click node_irq "https://github.com/scalex-io/ugds/blob/main/drv/irq.h"
click node_drvlist "https://github.com/scalex-io/ugds/blob/main/drv/list.h"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sync,node_batch,node_async,node_handle,node_ugdsstate toneBlue
class node_admin,node_controller,node_queue,node_nvm_dma,node_device,node_rpc,node_dma_internal,node_libctrl,node_regs,node_ssd,node_gpu toneAmber
class node_pci,node_map,node_drvctrl,node_irq,node_drvlist toneMint
class node_buffer,node_rdma,node_hugepage toneRose
class node_application,node_rdma_consumer toneIndigo
```

## Components

### uGDS API

- [Sync I/O](https://github.com/scalex-io/ugds/blob/main/src/ugds_io.cpp): `src/ugds_io.cpp`
- [Batch I/O](https://github.com/scalex-io/ugds/blob/main/src/ugds_batch.cpp): `src/ugds_batch.cpp`
- [Async I/O](https://github.com/scalex-io/ugds/blob/main/src/ugds_async.cpp): `src/ugds_async.cpp`
- [Handle Lifecycle](https://github.com/scalex-io/ugds/blob/main/src/ugds_handle.cpp): `src/ugds_handle.cpp`
- [uGDS State](https://github.com/scalex-io/ugds/blob/main/src/ugds_internal.h): `src/ugds_internal.h`

### NVMe Library

- [NVMe Admin](https://github.com/scalex-io/ugds/blob/main/src/libnvm/admin.cpp): `src/libnvm/admin.cpp`
- [Controller Setup](https://github.com/scalex-io/ugds/blob/main/src/libnvm/ctrl.cpp): `src/libnvm/ctrl.cpp`
- [Queue Management](https://github.com/scalex-io/ugds/blob/main/src/libnvm/queue.cpp): `src/libnvm/queue.cpp`
- [DMA Mapping](https://github.com/scalex-io/ugds/blob/main/src/libnvm/linux_dma.cpp): `src/libnvm/linux_dma.cpp`
- [Linux Device](https://github.com/scalex-io/ugds/blob/main/src/libnvm/linux_device.cpp): `src/libnvm/linux_device.cpp`
- [RPC](https://github.com/scalex-io/ugds/blob/main/src/libnvm/rpc.cpp): `src/libnvm/rpc.cpp`
- [DMA Descriptors](https://github.com/scalex-io/ugds/blob/main/src/libnvm/internal/dma.h): `src/libnvm/internal/dma.h`
- [Controller State](https://github.com/scalex-io/ugds/blob/main/src/libnvm/internal/lib_ctrl.h): `src/libnvm/internal/lib_ctrl.h`
- [NVMe Registers](https://github.com/scalex-io/ugds/blob/main/src/libnvm/internal/regs.h): `src/libnvm/internal/regs.h`

### Kernel Driver

- [PCI Driver](https://github.com/scalex-io/ugds/blob/main/drv/pci.c): `drv/pci.c`
- [GPU Page Mapping](https://github.com/scalex-io/ugds/blob/main/drv/map.c): `drv/map.c`
- [Driver Controller](https://github.com/scalex-io/ugds/blob/main/drv/ctrl.h): `drv/ctrl.h`
- [Interrupt Handling](https://github.com/scalex-io/ugds/blob/main/drv/irq.h): `drv/irq.h`
- [Driver Lists](https://github.com/scalex-io/ugds/blob/main/drv/list.h): `drv/list.h`

### GPU Memory

- [Buffer Registration](https://github.com/scalex-io/ugds/blob/main/src/ugds_buf.cpp): `src/ugds_buf.cpp`
- [RDMA Registration](https://github.com/scalex-io/ugds/blob/main/src/ugds_rdma.cpp): `src/ugds_rdma.cpp`
- [Hugepages](https://github.com/scalex-io/ugds/blob/main/src/ugds_hugepage.cpp): `src/ugds_hugepage.cpp`

### Other components

- **Application**
- **NVMe SSD**
- **GPU Memory**
- **RDMA Consumer**

## Connections

- Application → Handle Lifecycle: registers handle
- Application → Buffer Registration: registers buffer
- Application → Sync I/O: issues transfer
- Application → Batch I/O: submits batch
- Application → Async I/O: issues async
- Sync I/O → NVMe SSD: transfers data
- Sync I/O → GPU Memory: transfers data
- Application → RDMA Registration: registers memory
- RDMA Consumer → GPU Memory: uses exported memory
- Buffer Registration → DMA Mapping: maps GPU buffer (evidence: `src/ugds_buf.cpp`)
- DMA Mapping → DMA Descriptors: initializes mapping (evidence: `src/libnvm/linux_dma.cpp`)
- DMA Mapping → Controller State: uses controller (evidence: `src/libnvm/linux_dma.cpp`)
- Controller Setup → Controller State: uses state (evidence: `src/libnvm/ctrl.cpp`)
- NVMe Admin → NVMe Registers: accesses registers (evidence: `src/libnvm/admin.cpp`)
- NVMe Admin → RPC: uses RPC (evidence: `src/libnvm/admin.cpp`)
- RPC → Controller State: uses controller (evidence: `src/libnvm/rpc.cpp`)
- PCI Driver → Driver Controller: manages controller (evidence: `drv/pci.c`)
- PCI Driver → GPU Page Mapping: maps GPU pages (evidence: `drv/pci.c`)
- PCI Driver → Interrupt Handling: registers interrupts (evidence: `drv/pci.c`)
- GPU Page Mapping → Driver Lists: walks controller list (evidence: `drv/map.c`)
- Driver Controller → Driver Lists: stores list node (evidence: `drv/ctrl.h`)
- Async I/O → uGDS State: uses shared state (evidence: `src/ugds_async.cpp`)
- Batch I/O → uGDS State: uses shared state (evidence: `src/ugds_batch.cpp`)
- Sync I/O → uGDS State: uses shared state (evidence: `src/ugds_io.cpp`)
- PCI Driver → NVMe SSD: exposes PCI device
