How cinoadam/nvml-unified-shim works
This repo is a native Linux dynamic library shim that interposes NVML APIs via LD_PRELOAD to make NVML-dependent applications work on Grace Blackwell GB10 unified-memory systems where certain NVML queries return NVML_ERROR_NOT_SUPPORTED.
So the “system” is not a service topology; it’s a runtime call-flow architecture:
• A third-party Application links against NVML (libnvidia-ml.so)
• At runtime the loader injects our shim (libnvml-unified.so)
• The shim wraps selected NVML symbols and either:
• delegates to real NVML (via dlsym(RTLD_NEXT, ...)), or
• falls back to CUDA runtime (libcudart) + Linux /proc data sources
Top-level structure and what it implies:
• src/
• nvml_shim.c → the interposition layer: exports nvml* functions, loads “real” NVML symbols with dlsym, implements branching (try real NVML, fallback on NOT_SUPPORTED).
• nvml_unified.c → the unified-memory fallback implementation: reads /proc/meminfo, calls CUDA runtime APIs (cudaGetDeviceCount, cudaGetDeviceProperties, memory usage queries), formats NVML return structs.
• tests/
• test_python.py → indicates Python-based verification / harness (even if minimal).
NVML Client Application
• Examples: MAX Engine, nvtop, TensorRT tools
• Calls NVML C API functions (e.g., nvmlDeviceGetMemoryInfo)
Dynamic Loader / Interposition Mechanism
• LD_PRELOAD
• ELF symbol resolution order
Shim Library: libnvml-unified.so
• “NVML wrapper exports”
• “Dispatch layer” (dlsym(RTLD_NEXT, ...))
• “Fallback provider” (unified memory logic)
Real NVML: libnvidia-ml.so
• Returns success on supported platforms; returns NVML_ERROR_NOT_SUPPORTED on GB10 for memory info
CUDA Runtime: libcudart.so
• cudaGetDeviceCount, cudaGetDeviceProperties, plus memory usage queries (as implemented)
Linux Kernel Interfaces
• /proc/meminfo (system/unified memory total)
Hardware/Platform Context
• Grace Blackwell GB10 unified memory (shared CPU/GPU LPDDR5x)
• Debug logging gated by NVML_SHIM_DEBUG
• Tests (tests/test_basic and Python test) as “validation clients”
• App → (calls) → nvml*() symbol
• Loader resolves nvml*() → Shim (because LD_PRELOAD)
• Shim → (dlsym RTLD_NEXT) → Real NVML
• If error: Shim → (calls) → CUDA runtime
• Shim → (reads) → /proc/meminfo
• Shim → (returns NVML structs/return codes) → App
Make the decision point explicit: “if NVML returns NOT_SUPPORTED ⇒ fallback”.
Important architectural patterns to highlight (specific to this project)
• Dynamic library interposition (LD_PRELOAD)
• Adapter/Facade: exposes NVML API surface but sources data from CUDA + OS
• Progressive enhancement / fallback strategy: “try real NVML first”
• Partial API coverage: PoC only implements a subset; others likely pass-through or error
Also call out the fake handle strategy (index-as-pointer trick). This is an implementation detail but architecturally important because it affects compatibility.
Goal: show runtime link/interposition and dependencies.
• Notes: activation via LD_PRELOAD, optional system-wide install
Goal: show the core algorithm and decision branch.
Lifelines:
• Application
• Shim nvmlDeviceGetMemoryInfo
• Real NVML nvmlDeviceGetMemoryInfo (RTLD_NEXT)
• /proc/meminfo
• CUDA runtime
Sequence:
Shim computes free and returns NVML_SUCCESS with filled struct
An alternate path:
• If real NVML returns NVML_SUCCESS, shim returns immediately (pass-through)
Goal: make clear which functions are wrapped and how they’re satisfied.
• nvmlDeviceGetCount → NVML else CUDA
• nvmlDeviceGetHandleByIndex → NVML else fake handle
• nvmlDeviceGetMemoryInfo(v1/v2) → NVML else /proc + CUDA
• nvmlDeviceGetName → NVML else CUDA props
• (future/TODO rows) PCIe, thermals, clocks, process tracking…
Use consistent visual semantics:
• Rectangles for binaries/libraries:
• Application, libnvml-unified.so, libnvidia-ml.so, libcudart.so
• /proc/meminfo
• Hexagon or cloud for “platform/hardware”:
• GB10 unified memory architecture
• Diamonds for decision points:
• “NVML supported?” / “ret == NOT_SUPPORTED?”
• Blue: caller/application
• Orange: shim/interposition logic
• Green: “official” vendor libs (NVML/CUDA)
• Gray: OS/kernel interfaces (/proc)
• Red callouts: limitations (fake handles, partial coverage, incorrect PCIe metrics)
• Always label with the function or mechanism:
• nvmlDeviceGetMemoryInfo()
• dlsym(RTLD_NEXT, "nvmlDeviceGetMemoryInfo")
• cudaGetDeviceCount()
• read /proc/meminfo
Final “best representation” framing (what to title it)
Title suggestion:
“NVML Unified Memory Shim — Runtime Interposition & Fallback Architecture (GB10)”
Subtitle:
“LD_PRELOAD NVML wrapper that falls back to CUDA runtime + /proc when NVML reports NOT_SUPPORTED.”
That title prevents misinterpretation as a service architecture and correctly centers the runtime mechanism.