# cinoadam/nvml-unified-shim architecture

> Architecture diagram of the GitHub repository cinoadam/nvml-unified-shim, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cinoadam/nvml-unified-shim
- Repository: https://github.com/cinoadam/nvml-unified-shim

## Overview

This repo is a **native Linux dynamic library shim** that **interposes NVML APIs** via **`LD_PRELOAD`** to make **NVML-dependent applications work on Grace Blackwell GB10 unified-memory systems** where certain NVML queries return `NVML_ERROR_NOT_SUPPORTED`.

So the “system” is not a service topology; it’s a **runtime call-flow architecture**:
- A third-party **Application** links against **NVML (`libnvidia-ml.so`)**
- At runtime the loader injects **our shim** (`libnvml-unified.so`)
- The shim **wraps** selected NVML symbols and either:
- delegates to **real NVML** (via `dlsym(RTLD_NEXT, ...)`), or
- falls back to **CUDA runtime (`libcudart`)** + **Linux `/proc`** data sources

Top-level structure and what it implies:

- `src/`
- `nvml_shim.c` → the **interposition layer**: exports `nvml*` functions, loads “real” NVML symbols with `dlsym`, implements branching (try real NVML, fallback on NOT_SUPPORTED).
- `nvml_unified.c` → the **unified-memory fallback implementation**: reads `/proc/meminfo`, calls CUDA runtime APIs (`cudaGetDeviceCount`, `cudaGetDeviceProperties`, memory usage queries), formats NVML return structs.
- `tests/`
- `test_python.py` → indicates Python-based verification / harness (even if minimal).

##### NVML Client Application
- Examples: MAX Engine, nvtop, TensorRT tools
- Calls NVML C API functions (e.g., `nvmlDeviceGetMemoryInfo`)
##### Dynamic Loader / Interposition Mechanism
- `LD_PRELOAD`
- ELF symbol resolution order
##### Shim Library: `libnvml-unified.so`
- “NVML wrapper exports”
- “Dispatch layer” (`dlsym(RTLD_NEXT, ...)`)
- “Fallback provider” (unified memory logic)
##### Real NVML: `libnvidia-ml.so`
- Returns success on supported platforms; returns `NVML_ERROR_NOT_SUPPORTED` on GB10 for memory info
##### CUDA Runtime: `libcudart.so`
- `cudaGetDeviceCount`, `cudaGetDeviceProperties`, plus memory usage queries (as implemented)
##### Linux Kernel Interfaces
- `/proc/meminfo` (system/unified memory total)
##### Hardware/Platform Context
- Grace Blackwell GB10 unified memory (shared CPU/GPU LPDDR5x)

- **Debug logging** gated by `NVML_SHIM_DEBUG`
- **Tests** (`tests/test_basic` and Python test) as “validation clients”

- App → (calls) → `nvml*()` symbol
- Loader resolves `nvml*()` → Shim (because `LD_PRELOAD`)
- Shim → (dlsym RTLD_NEXT) → Real NVML
- If error: Shim → (calls) → CUDA runtime
- Shim → (reads) → `/proc/meminfo`
- Shim → (returns NVML structs/return codes) → App

Make the decision point explicit: **“if NVML returns NOT_SUPPORTED ⇒ fallback”**.

##### Important architectural patterns to highlight (specific to this project)

- **Dynamic library interposition (LD_PRELOAD)**
- **Adapter/Facade**: exposes NVML API surface but sources data from CUDA + OS
- **Progressive enhancement / fallback strategy**: “try real NVML first”
- **Partial API coverage**: PoC only implements a subset; others likely pass-through or error

Also call out the **fake handle strategy** (index-as-pointer trick). This is an implementation detail but architecturally important because it affects compatibility.

**Goal:** show runtime link/interposition and dependencies.

- Notes: activation via `LD_PRELOAD`, optional system-wide install

**Goal:** show the core algorithm and decision branch.

Lifelines:
- Application
- Shim `nvmlDeviceGetMemoryInfo`
- Real NVML `nvmlDeviceGetMemoryInfo` (RTLD_NEXT)
- `/proc/meminfo`
- CUDA runtime

Sequence:
##### Shim computes free and returns `NVML_SUCCESS` with filled struct

An alternate path:
- If real NVML returns `NVML_SUCCESS`, shim returns immediately (pass-through)

**Goal:** make clear which functions are wrapped and how they’re satisfied.

- `nvmlDeviceGetCount` → NVML else CUDA
- `nvmlDeviceGetHandleByIndex` → NVML else fake handle
- `nvmlDeviceGetMemoryInfo(v1/v2)` → NVML else `/proc` + CUDA
- `nvmlDeviceGetName` → NVML else CUDA props
- (future/TODO rows) PCIe, thermals, clocks, process tracking…

Use consistent visual semantics:

- **Rectangles** for binaries/libraries:
- Application, `libnvml-unified.so`, `libnvidia-ml.so`, `libcudart.so`
- `/proc/meminfo`
- **Hexagon** or **cloud** for “platform/hardware”:
- GB10 unified memory architecture
- **Diamonds** for decision points:
- “NVML supported?” / “ret == NOT_SUPPORTED?”

- Blue: caller/application
- Orange: shim/interposition logic
- Green: “official” vendor libs (NVML/CUDA)
- Gray: OS/kernel interfaces (`/proc`)
- Red callouts: limitations (fake handles, partial coverage, incorrect PCIe metrics)

- Always label with the function or mechanism:
- `nvmlDeviceGetMemoryInfo()`
- `dlsym(RTLD_NEXT, "nvmlDeviceGetMemoryInfo")`
- `cudaGetDeviceCount()`
- `read /proc/meminfo`

##### Final “best representation” framing (what to title it)
Title suggestion:
**“NVML Unified Memory Shim — Runtime Interposition & Fallback Architecture (GB10)”**

Subtitle:
**“LD_PRELOAD NVML wrapper that falls back to CUDA runtime + /proc when NVML reports NOT_SUPPORTED.”**

That title prevents misinterpretation as a service architecture and correctly centers the runtime mechanism.

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =========================================================
  %% Diagram 1 — Runtime Component / Dependency Diagram
  %% =========================================================
  subgraph "Diagram 1 — Runtime Interposition & Dependencies (GB10 Unified Memory)"
    direction TB

    App["NVML Client Application\n(e.g., MAX Engine, nvtop, TensorRT tools)"]:::app
    Loader["Dynamic Loader / ELF Symbol Resolution\n(LD_PRELOAD)"]:::loader

    subgraph "Shim Library: libnvml-unified.so (Injected)"
      direction TB
      ShimExports["NVML Wrapper Exports\n(nvml* symbols)"]:::shim
      Dispatch["Dispatch Layer\n(dlsym(RTLD_NEXT, ...))"]:::shim
      Fallback["Unified-Memory Fallback Provider\n(CUDA runtime + /proc parsing)"]:::shim
      Debug["Debug Logging\n(NVML_SHIM_DEBUG=1)"]:::obs
    end

    RealNVML["Real NVML Library\n(libnvidia-ml.so)"]:::vendor
    CUDART["CUDA Runtime\n(libcudart.so)"]:::vendor
    ProcMem[("/proc/meminfo\n(total unified/system memory)")]:::os
    GB10{"Grace Blackwell GB10\nUnified Memory Platform"}:::hw

    App -->|"calls nvml*()"| Loader
    Loader -->|"resolves symbols via LD_PRELOAD"| ShimExports
    ShimExports -->|"delegates (pass-through)"| Dispatch
    Dispatch -->|"dlsym(RTLD_NEXT, 'nvml*')"| RealNVML

    RealNVML -->|"returns NVML_SUCCESS"| ShimExports
    RealNVML -->|"returns NVML_ERROR_NOT_SUPPORTED"| Fallback

    Fallback -->|"calls CUDA runtime APIs"| CUDART
    Fallback -->|"reads"| ProcMem
    ProcMem -->|"provided by"| GB10
    CUDART -->|"queries GPU properties/usage"| GB10

    Fallback -->|"returns filled NVML structs\n+ NVML_SUCCESS"| ShimExports
    ShimExports -->|"returns status + data"| App

    ShimExports -.->|"optional logs"| Debug
    Fallback -.->|"optional logs"| Debug
  end

  %% =========================================================
  %% Diagram 2 — Sequence (Flow) for nvmlDeviceGetMemoryInfo()
  %% =========================================================
  subgraph "Diagram 2 — Call Flow: nvmlDeviceGetMemoryInfo() (Primary Fallback Path)"
    direction TB

    A2["Application"]:::app
    S2["Shim: nvmlDeviceGetMemoryInfo()\n(interposed symbol)"]:::shim
    N2["Real NVML: nvmlDeviceGetMemoryInfo()\n(RTLD_NEXT)"]:::vendor
    D2{"ret == NVML_ERROR_NOT_SUPPORTED?"}:::decision
    P2[("/proc/meminfo\nread total memory")]:::os
    C2["CUDA Runtime\n(query used memory)"]:::vendor
    R2["Return to Application\n(NVML_SUCCESS + struct)"]:::return

    A2 -->|"nvmlDeviceGetMemoryInfo()"| S2
    S2 -->|"dlsym(RTLD_NEXT) + call"| N2
    N2 -->|"ret + (maybe) struct"| S2
    S2 -->|"check ret"| D2

    D2 -->|"No (NVML_SUCCESS)"| R2
    D2 -->|"Yes (NOT_SUPPORTED)"| P2
    P2 -->|"total"| S2
    S2 -->|"cuda memory usage queries"| C2
    C2 -->|"used (and device info as needed)"| S2
    S2 -->|"compute free = total - used\nfill nvmlMemory_t / v2"| R2
    R2 -->|"status + data"| A2
  end

  %% =========================================================
  %% Diagram 3 — API Surface Coverage / Routing Table
  %% =========================================================
  subgraph "Diagram 3 — Wrapped NVML API Surface (Routing / Fallback Table)"
    direction TB

    T0["NVML API Call (from app)"]:::tableHead

    subgraph "Device Enumeration"
      direction TB
      T1["nvmlDeviceGetCount()"]:::api
      T1a["Try Real NVML\nelse cudaGetDeviceCount()"]:::route
      T2["nvmlDeviceGetHandleByIndex()"]:::api
      T2a["Try Real NVML\nelse fake handle (index-as-pointer)"]:::route
    end

    subgraph "Device Identity"
      direction TB
      T3["nvmlDeviceGetName()"]:::api
      T3a["Try Real NVML\nelse cudaGetDeviceProperties()"]:::route
    end

    subgraph "Memory (Unified Memory Fix)"
      direction TB
      T4["nvmlDeviceGetMemoryInfo()"]:::api
      T4a["Try Real NVML\nelse /proc/meminfo + CUDA usage"]:::route
      T5["nvmlDeviceGetMemoryInfo_v2()"]:::api
      T5a["Try Real NVML\nelse /proc/meminfo + CUDA usage"]:::route
    end

    subgraph "Notes / Limitations"
      direction TB
      L1["Partial NVML coverage (PoC)\nOther APIs: pass-through or unsupported"]:::note
      L2["Fake handle strategy impacts compatibility\n(handle is not a real NVML object)"]:::note
    end

    T0 -->|"dispatch"| T1 --> T1a
    T0 -->|"dispatch"| T2 --> T2a
    T0 -->|"dispatch"| T3 --> T3a
    T0 -->|"dispatch"| T4 --> T4a
    T0 -->|"dispatch"| T5 --> T5a

    T2a -.->|"implementation detail"| L2
    T4a -.->|"primary fix for GB10 unified memory"| L1
  end

  %% =========================================================
  %% Validation / Build / Docs / Utilities (Adjunct)
  %% =========================================================
  subgraph "Repo Artifacts (Build, Tests, Docs, Utilities)"
    direction TB
    Build["Build & Install\n(Makefile / Makefile.python)\nProduces libnvml-unified.so\nActivated via LD_PRELOAD"]:::build
    Doc["API Reference\n(Wrapped functions & behavior)"]:::doc
    TestC["Validation Client (C)\nNVML consumer for call-flow"]:::test
    TestPy["Validation Client (Python)\nverification / harness"]:::test
    Util["Optional Adjunct Utilities\n(gpu-info, nvidia-smi-wrapper)"]:::util

    Build -->|"installs .so & usage guidance"| Doc
    TestC -->|"calls nvml*()"| App
    TestPy -->|"calls into NVML stack"| App
    Util -.->|"optional usage"| App
  end

  %% =========================================================
  %% Click Events (from component_mapping)
  %% =========================================================
  click ShimExports "https://github.com/cinoadam/nvml-unified-shim/blob/main/src/nvml_shim.c"
  click Dispatch "https://github.com/cinoadam/nvml-unified-shim/blob/main/src/nvml_shim.c"
  click Fallback "https://github.com/cinoadam/nvml-unified-shim/blob/main/src/nvml_unified.c"
  click Doc "https://github.com/cinoadam/nvml-unified-shim/blob/main/API_REFERENCE.md"
  click Build "https://github.com/cinoadam/nvml-unified-shim/tree/main/Makefile"
  click Build "https://github.com/cinoadam/nvml-unified-shim/blob/main/Makefile.python"
  click TestC "https://github.com/cinoadam/nvml-unified-shim/blob/main/tests/test_basic.c"
  click TestC "https://github.com/cinoadam/nvml-unified-shim/tree/main/tests/test_basic"
  click TestPy "https://github.com/cinoadam/nvml-unified-shim/blob/main/tests/test_python.py"
  click Util "https://github.com/cinoadam/nvml-unified-shim/tree/main/gpu-info"
  click Util "https://github.com/cinoadam/nvml-unified-shim/tree/main/nvidia-smi-wrapper"

  %% =========================================================
  %% Styles
  %% =========================================================
  classDef app fill:#1f77b4,stroke:#0b3d66,color:#ffffff,stroke-width:1.5px
  classDef loader fill:#4aa3df,stroke:#0b3d66,color:#ffffff,stroke-width:1.5px
  classDef shim fill:#ff8c1a,stroke:#a65100,color:#1a1a1a,stroke-width:1.5px
  classDef vendor fill:#2ca02c,stroke:#145214,color:#ffffff,stroke-width:1.5px
  classDef os fill:#9e9e9e,stroke:#424242,color:#ffffff,stroke-width:1.5px
  classDef hw fill:#6a5acd,stroke:#2f2a7a,color:#ffffff,stroke-width:1.5px
  classDef decision fill:#ffd54f,stroke:#8a6d00,color:#1a1a1a,stroke-width:1.5px
  classDef obs fill:#ffccbc,stroke:#bf360c,color:#1a1a1a,stroke-width:1.5px
  classDef return fill:#c8e6c9,stroke:#1b5e20,color:#1a1a1a,stroke-width:1.5px

  classDef build fill:#8d6e63,stroke:#3e2723,color:#ffffff,stroke-width:1.5px
  classDef doc fill:#7b1fa2,stroke:#4a0072,color:#ffffff,stroke-width:1.5px
  classDef test fill:#00acc1,stroke:#006064,color:#ffffff,stroke-width:1.5px
  classDef util fill:#455a64,stroke:#1c313a,color:#ffffff,stroke-width:1.5px

  classDef api fill:#1565c0,stroke:#0d47a1,color:#ffffff,stroke-width:1.5px
  classDef route fill:#ffb74d,stroke:#e65100,color:#1a1a1a,stroke-width:1.5px
  classDef note fill:#ffcdd2,stroke:#b71c1c,color:#1a1a1a,stroke-width:1.5px
  classDef tableHead fill:#0d47a1,stroke:#082f66,color:#ffffff,stroke-width:1.5px
```
