# nvidia/nvbench architecture

> Architecture diagram of the GitHub repository nvidia/nvbench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia/nvbench
- Repository: https://github.com/nvidia/nvbench

## Overview

##### Project Type and Purpose
- NVBench is a C++17 library plus CLI tool and Python extension for benchmarking CUDA kernels (and CPU code).
- It provides build-time parameter sweeps, runtime CLI configuration, multiple output formats, and hooks into CUDA runtime, CUPTI, NVML.
- There is also a Python package wrapping the core C++ library and standalone scripts for post-processing results.

##### High-Level Components
A. Build System (CMake + Presets)
- CMakeLists.txt at project root
- cmake/ (helper modules, dependency detection, install rules)
- CMakePresets.json (standardized build configurations)
B. Core Library (nvbench/)
- nvbench/*.cu / .cuh / .cxx: benchmark framework, axes, state management, timing, printing
- nvbench/detail, nvbench/internal: low-level utilities, kernel launch wrappers, statistics
- nvbench main entrypoints: launcher, CLI parser, printer plugins (markdown, CSV, JSON)
C. CLI Executable & Examples
- exec/nvbench-ctl.cu: example control utility (optional)
- examples/: sample benchmarks demonstrating axes, timers, streams, skipping, summaries
D. Python Layer
- python/src/py_nvbench.cpp: pybind11 wrapper exposing C++ API
- python/cuda/bench/: pure-Python front end exposing benchmark API
- python/examples: usage samples (auto-throughput, PyTorch bench, etc.)
- pyproject.toml, CMakeLists.txt: build Python wheel with C++ extension
E. Testing & Validation
- testing/: CUDA-based unit tests for each feature (axes, criteria, streams)
- python/test/: Python-level tests for the wrapper
- CI scripts under ci/ and .github/workflows: matrix builds across compilers, CUDA versions, verification of devcontainers
F. Dev Environment
- .devcontainer/: Docker-based dev images for various CUDA + compiler combos, entrypoint scripts, launch scripts, auto-verification

##### Relationships and Interactions
- The Build System (CMake) configures and compiles:
- Core Library → static or shared lib
- CLI Executable linked against Core Library
- Examples linked against Core Library
- Python Extension (py_nvbench) built via CMake + pybind11 → links to Core Library
- Tests built and registered with CTest
- At runtime:
- CLI Executable loads GPU drivers, enumerates devices via CUDA Runtime & NVML
- Core Library orchestrates kernel launches, times them with either CUDA Events or CUPTI, collects statistics
- Printers output through selected plugins (markdown, CSV, JSON)
- Python front end invokes the same C++ core under the hood
- Post-processing scripts (scripts/) read JSON or CSV outputs to generate histograms, comparisons
- CI/CD pipelines spin up devcontainers → run CMake configure + build + test across multiple CUDA/toolchain combos → publish artifacts or wheel

##### Architectural Patterns and Design Principles
- Modular CMake structure: separation of dependency detection, install rules, test registration
- Plugin pattern for output serializers (printer_base, printer_multiplex)
- Axis abstraction for parameter sweeps (static vs runtime)
- Single-responsibility: state management vs kernel launching vs printing vs parsing
- Python binding via pybind11 encapsulating core in a thin layer

##### Detailed Capture
- Don’t omit submodules: show nvbench/detail and nvbench/internal as child nodes under Core Library
- Python/src and python/cuda as distinct layers of the Python binding
- How .devcontainer images map to matrix builds in CI
- Tests as a separate consumer of the Core Library via CTest

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Build & Dev Environment
    subgraph "Build & Dev Environment"
        direction TB
        CMakeRoot["CMakeLists.txt"] 
        Presets["CMakePresets.json"]
        CMakeModules["cmake/"]
        DevContainer[".devcontainer/"]
    end

    %% Core Components
    subgraph "Core Components"
        direction TB
        CoreLibrary["Core Library (nvbench/)"]
        Detail["nvbench/detail/"]
        Internal["nvbench/internal/"]
        CLIExec["CLI Executable (nvbench)"]
        ControlUtil["Example Control (exec/nvbench-ctl.cu)"]
        Examples["Example Benchmarks (examples/)"]
        PyExtension["Python Extension"]
        PySrc["python/src/py_nvbench.cpp"]
        PyCMake["python/CMakeLists.txt"]
        PyProjectRoot["pyproject.toml"]
        PyProject["python/pyproject.toml"]
        PyFrontend["python/cuda/bench/"]
        PyExamples["python/examples/"]
        PyTests["python/test/"]
        TestingHarness["Testing Harness"]
        Testing["testing/"]
        TestingCMake["testing/cmake/"]
        Scripts["Post-processing Scripts (scripts/)"]
        ScriptsJSON["scripts/nvbench_json/"]
    end

    %% External Systems
    subgraph "External Systems"
        direction TB
        CUDARuntime["CUDA Runtime"]
        CUPTI["CUPTI"]
        NVML["NVML"]
        GPU["GPU Hardware"]
        PythonInterp["Python Interpreter"]
    end

    %% CI/CD
    subgraph "CI/CD Pipeline"
        direction TB
        CICommon["ci/"]
        GHWorkflows[".github/workflows/"]
        GHActions[".github/actions/compute-matrix"]
    end

    %% Build relationships
    CMakeRoot -->|"includes modules"| CMakeModules
    CMakeRoot --> Presets
    CMakeRoot --> CoreLibrary
    CMakeRoot --> CLIExec
    CMakeRoot --> Examples
    CMakeRoot --> PyExtension
    CMakeRoot --> TestingHarness
    CMakeModules --> CoreLibrary
    Presets --> DevContainer

    %% Core internal structure
    CoreLibrary --> Detail
    CoreLibrary --> Internal
    CLIExec --> CoreLibrary
    ControlUtil --> CoreLibrary
    Examples --> CoreLibrary
    PyExtension --> CoreLibrary
    PyExtension --> PySrc
    PyExtension --> PyCMake
    PyExtension --> PyProject
    PySrc --> PyProjectRoot
    PyFrontend --> PyExtension
    PyExamples --> PyFrontend
    PyTests --> PyFrontend
    TestingHarness --> CoreLibrary
    TestingHarness --> Testing
    TestingHarness --> TestingCMake
    Scripts --> ScriptsJSON

    %% Runtime interactions
    CoreLibrary -->|"CUDA API"| CUDARuntime
    CoreLibrary -->|"CUPTI API"| CUPTI
    CoreLibrary -->|"NVML API"| NVML
    CUDARuntime --> GPU
    CUPTI --> GPU
    NVML --> GPU
    CLIExec -->|"runtime"| PythonInterp
    PyFrontend --> PythonInterp
    CLIExec -- "outputs (JSON/CSV/MD)" --> Scripts

    %% CI/CD to Build & Dev
    CICommon --> CMakeRoot
    CICommon --> DevContainer
    CICommon --> GHWorkflows
    CICommon --> GHActions

    %% Styles
    classDef build fill:#ADD8E6,stroke:#000;
    classDef core fill:#90EE90,stroke:#000;
    classDef ext fill:#FFD580,stroke:#000;
    classDef ci fill:#D8BFD8,stroke:#000;

    class CMakeRoot,Presets,CMakeModules,DevContainer build;
    class CoreLibrary,Detail,Internal,CLIExec,ControlUtil,Examples,PyExtension,PySrc,PyCMake,PyProjectRoot,PyProject,PyFrontend,PyExamples,PyTests,TestingHarness,Testing,TestingCMake,Scripts,ScriptsJSON core;
    class CUDARuntime,CUPTI,NVML,GPU,PythonInterp ext;
    class CICommon,GHWorkflows,GHActions ci;

    %% Click Events
    click CMakeRoot "https://github.com/nvidia/nvbench/blob/main/CMakeLists.txt"
    click Presets "https://github.com/nvidia/nvbench/blob/main/CMakePresets.json"
    click CMakeModules "https://github.com/nvidia/nvbench/tree/main/cmake/"
    click DevContainer "https://github.com/nvidia/nvbench/tree/main/.devcontainer/"
    click CoreLibrary "https://github.com/nvidia/nvbench/tree/main/nvbench/"
    click Detail "https://github.com/nvidia/nvbench/tree/main/nvbench/detail/"
    click Internal "https://github.com/nvidia/nvbench/tree/main/nvbench/internal/"
    click CLIExec "https://github.com/nvidia/nvbench/blob/main/nvbench/main.cu"
    click ControlUtil "https://github.com/nvidia/nvbench/blob/main/exec/nvbench-ctl.cu"
    click Examples "https://github.com/nvidia/nvbench/tree/main/examples/"
    click PySrc "https://github.com/nvidia/nvbench/blob/main/python/src/py_nvbench.cpp"
    click PyCMake "https://github.com/nvidia/nvbench/blob/main/python/CMakeLists.txt"
    click PyProjectRoot "https://github.com/nvidia/nvbench/blob/main/pyproject.toml"
    click PyProject "https://github.com/nvidia/nvbench/blob/main/python/pyproject.toml"
    click PyFrontend "https://github.com/nvidia/nvbench/tree/main/python/cuda/bench/"
    click PyExamples "https://github.com/nvidia/nvbench/tree/main/python/examples/"
    click PyTests "https://github.com/nvidia/nvbench/tree/main/python/test/"
    click Scripts "https://github.com/nvidia/nvbench/tree/main/scripts/"
    click ScriptsJSON "https://github.com/nvidia/nvbench/tree/main/scripts/nvbench_json/"
    click Testing "https://github.com/nvidia/nvbench/tree/main/testing/"
    click TestingCMake "https://github.com/nvidia/nvbench/tree/main/testing/cmake/"
    click CICommon "https://github.com/nvidia/nvbench/tree/main/ci/"
    click GHWorkflows "https://github.com/nvidia/nvbench/tree/main/.github/workflows/"
    click GHActions "https://github.com/nvidia/nvbench/tree/main/.github/actions/compute-matrix"
```
