# dandinpower/numa-allocation architecture

> Architecture diagram of the GitHub repository dandinpower/numa-allocation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dandinpower/numa-allocation
- Repository: https://github.com/dandinpower/numa-allocation

## Overview

##### Project type and purpose
- This is an open-source Python package (“NUMA-Aware Tensor Allocation for PyTorch”) that exposes a NUMA-aware CPU memory allocator to PyTorch users.
- It wraps a small CUDA/C++ extension that in turn uses Linux NUMA libraries (`libnuma`) and optionally pins pages for faster host-to-device transfers.

##### File structure
- Top-level files:
- setup.py – build script for packaging and compiling the CUDA extension
- requirements.txt – Python dependencies (e.g., torch)
- README.md – project description, usage
- test_correctness.py – a driver that calls the API and inspects NUMA placement via `numastat`
- src/numa_allocation:
- allocation.cu – CUDA/C++ code implementing the allocator and optional page-pinning
- wrapper.py – Python module exposing allocation functions (`zeros_numa_onnode_cpu`, etc.)

##### Extract architecture clues from the README
- Dependencies on `libnuma-dev`, `numactl`, CUDA Toolkit (nvcc)
- Packaging via pip (setup.py builds the CUDA extension)
- Python API sits atop the extension; uses PyTorch Tensors with pinned host memory

##### Main components and their interactions
a. User Code / PyTorch Application
– Imports the Python API (`from numa_allocation import zeros_numa_onnode_cpu`)
b. Python Wrapper (`wrapper.py`)
– Defines the high-level API
– Calls into the compiled extension via PyBind11 or CPython C API
c. CUDA Extension (`allocation.cu`)
– Implements `zeros_numa_*` functions
– Uses `numa_alloc_onnode`, `numa_alloc_interleaved` from `libnuma`
– Optionally invokes `cudaHostRegister` to pin pages for CUDA transfers
d. Underlying Libraries and OS
– Linux NUMA subsystem (libnuma)
– CUDA runtime and driver
– numactl (for testing)
e. Build and Packaging
– setup.py invokes nvcc and setuptools to compile allocation.cu into a Python module

##### Architectural patterns and design principles
- Layered architecture:
[User code] → [Python wrapper/API] → [Native CUDA/C++ extension] → [OS libraries]
- Separation of concerns: allocation logic (native layer) vs. Python interface
- Extensibility: additional allocation strategies can be added as new exported functions
- Test harness integrates with system utilities (numactl, numastat) for validation

a. Components:
- Client application (PyTorch code)
- Python API layer (`wrapper.py`)
- Native extension module (compiled from `allocation.cu`)
- libnuma (user-space NUMA allocator)
- CUDA runtime (for pinning memory)
- OS-level utilities (numactl / numastat)
- Build toolchain (nvcc, setuptools)
b. Relationships / Data flow:
1. Client → calls API in wrapper.py
2. Wrapper → invokes native extension functions
3. Extension → calls libnuma for allocation
4. Extension → optionally calls CUDA runtime to pin memory
5. Extension → returns a raw buffer to PyTorch as a CPU tensor (with .is_pinned)
6. Client → uses the tensor in GPU kernels or data transfers
7. Test harness → runs API under numactl, inspects via numastat
c. Key technologies:
- Python 3.12, PyTorch
- CUDA & nvcc
- libnuma, numactl/numastat
- setuptools / pip
d. Patterns: layered FFI, resource management (pin/unpin), test-driven validation

##### Summary of essential elements to capture
- Explicit layering from Python → native → OS
- The dual role of the extension (NUMA allocation + CUDA pinning)
- Build-time vs. run-time components
- Testing harness as an external integration

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User Layer"
        direction TB
        TestHarness["Test Harness / Client Application"]:::user
        README["Project Documentation<br/>(README.md)"]:::user
    end

    %% Python Layer
    subgraph "Python Layer"
        direction TB
        EntryPoint["Package Entry Point<br/>(__init__.py)"]:::python
        Wrapper["Python API Wrapper<br/>(wrapper.py)"]:::python
    end

    %% Native Extension Layer
    subgraph "Native/CUDA Extension"
        direction TB
        Extension["CUDA/C++ Extension<br/>(allocation.cu)"]:::native
    end

    %% OS / Libraries Layer
    subgraph "OS & Libraries"
        direction TB
        Libnuma["libnuma"]:::os
        CUDARuntime["CUDA Runtime"]:::os
        Numactl["numactl / numastat"]:::os
    end

    %% Build & Toolchain Layer
    subgraph "Build & Packaging"
        direction TB
        Setup["Build Script<br/>(setup.py)"]:::toolchain
        Requirements["Dependencies Manifest<br/>(requirements.txt)"]:::toolchain
    end

    %% Data Flows and Build Flow
    Setup -->|"compiles with nvcc & setuptools"| Extension
    Requirements --> Setup

    TestHarness -->|"API call"| Wrapper
    Wrapper -->|"calls native functions"| Extension
    Extension -->|"numa_alloc_onnode / interleaved"| Libnuma
    Extension -->|"cudaHostRegister"| CUDARuntime
    Extension -->|"returns pinned Tensor"| TestHarness
    TestHarness -->|"runs under"| Numactl
    TestHarness -->|"inspects placement via"| Numactl

    %% Click Events
    click TestHarness "https://github.com/dandinpower/numa-allocation/blob/main/test_correctness.py"
    click README "https://github.com/dandinpower/numa-allocation/blob/main/README.md"
    click EntryPoint "https://github.com/dandinpower/numa-allocation/blob/main/src/numa_allocation/__init__.py"
    click Wrapper "https://github.com/dandinpower/numa-allocation/blob/main/src/numa_allocation/wrapper.py"
    click Extension "https://github.com/dandinpower/numa-allocation/blob/main/src/numa_allocation/allocation.cu"
    click Setup "https://github.com/dandinpower/numa-allocation/blob/main/setup.py"
    click Requirements "https://github.com/dandinpower/numa-allocation/blob/main/requirements.txt"

    %% Styles
    classDef user fill:#a8f0c6,stroke:#333,stroke-width:1px;
    classDef python fill:#c6d8ff,stroke:#333,stroke-width:1px;
    classDef native fill:#ffd1a8,stroke:#333,stroke-width:1px;
    classDef os fill:#cccccc,stroke:#333,stroke-width:1px;
    classDef toolchain fill:#e0c3ff,stroke:#333,stroke-width:1px;
```
