How dandinpower/numa-allocation works
Project type and purpose
• This is an open-source Python package (“NUMA-Aware Tensor Allocation for PyTorch”) that exposes a NUMA-aware CPU memory allocator to PyTorch users.
• It wraps a small CUDA/C++ extension that in turn uses Linux NUMA libraries (libnuma) and optionally pins pages for faster host-to-device transfers.
File structure
• Top-level files:
• setup.py – build script for packaging and compiling the CUDA extension
• requirements.txt – Python dependencies (e.g., torch)
• README.md – project description, usage
• test_correctness.py – a driver that calls the API and inspects NUMA placement via numastat
• src/numa_allocation:
• allocation.cu – CUDA/C++ code implementing the allocator and optional page-pinning
• wrapper.py – Python module exposing allocation functions (zeros_numa_onnode_cpu, etc.)
Extract architecture clues from the README
• Dependencies on libnuma-dev, numactl, CUDA Toolkit (nvcc)
• Packaging via pip (setup.py builds the CUDA extension)
• Python API sits atop the extension; uses PyTorch Tensors with pinned host memory
Main components and their interactions
a. User Code / PyTorch Application
– Imports the Python API (from numa_allocation import zeros_numa_onnode_cpu)
b. Python Wrapper (wrapper.py)
– Defines the high-level API
– Calls into the compiled extension via PyBind11 or CPython C API
c. CUDA Extension (allocation.cu)
– Implements zeros_numa_* functions
– Uses numa_alloc_onnode, numa_alloc_interleaved from libnuma
– Optionally invokes cudaHostRegister to pin pages for CUDA transfers
d. Underlying Libraries and OS
– Linux NUMA subsystem (libnuma)
– CUDA runtime and driver
– numactl (for testing)
e. Build and Packaging
– setup.py invokes nvcc and setuptools to compile allocation.cu into a Python module
Architectural patterns and design principles
• Layered architecture:
[User code] → [Python wrapper/API] → [Native CUDA/C++ extension] → [OS libraries]
• Separation of concerns: allocation logic (native layer) vs. Python interface
• Extensibility: additional allocation strategies can be added as new exported functions
• Test harness integrates with system utilities (numactl, numastat) for validation
a. Components:
• Client application (PyTorch code)
• Python API layer (wrapper.py)
• Native extension module (compiled from allocation.cu)
• libnuma (user-space NUMA allocator)
• CUDA runtime (for pinning memory)
• OS-level utilities (numactl / numastat)
• Build toolchain (nvcc, setuptools)
b. Relationships / Data flow:
1. Client → calls API in wrapper.py
2. Wrapper → invokes native extension functions
3. Extension → calls libnuma for allocation
4. Extension → optionally calls CUDA runtime to pin memory
5. Extension → returns a raw buffer to PyTorch as a CPU tensor (with .is_pinned)
6. Client → uses the tensor in GPU kernels or data transfers
7. Test harness → runs API under numactl, inspects via numastat
c. Key technologies:
• Python 3.12, PyTorch
• CUDA & nvcc
• libnuma, numactl/numastat
• setuptools / pip
d. Patterns: layered FFI, resource management (pin/unpin), test-driven validation
Summary of essential elements to capture
• Explicit layering from Python → native → OS
• The dual role of the extension (NUMA allocation + CUDA pinning)
• Build-time vs. run-time components
• Testing harness as an external integration