How nvidia/nvbench works
Project Type and Purpose
• NVBench is a C++17 library plus CLI tool and Python extension for benchmarking CUDA kernels (and CPU code).
• It provides build-time parameter sweeps, runtime CLI configuration, multiple output formats, and hooks into CUDA runtime, CUPTI, NVML.
• There is also a Python package wrapping the core C++ library and standalone scripts for post-processing results.
High-Level Components
A. Build System (CMake + Presets)
• CMakeLists.txt at project root
• cmake/ (helper modules, dependency detection, install rules)
• CMakePresets.json (standardized build configurations)
B. Core Library (nvbench/)
• nvbench/*.cu / .cuh / .cxx: benchmark framework, axes, state management, timing, printing
• nvbench/detail, nvbench/internal: low-level utilities, kernel launch wrappers, statistics
• nvbench main entrypoints: launcher, CLI parser, printer plugins (markdown, CSV, JSON)
C. CLI Executable & Examples
• exec/nvbench-ctl.cu: example control utility (optional)
• examples/: sample benchmarks demonstrating axes, timers, streams, skipping, summaries
D. Python Layer
• python/src/py_nvbench.cpp: pybind11 wrapper exposing C++ API
• python/cuda/bench/: pure-Python front end exposing benchmark API
• python/examples: usage samples (auto-throughput, PyTorch bench, etc.)
• pyproject.toml, CMakeLists.txt: build Python wheel with C++ extension
E. Testing & Validation
• testing/: CUDA-based unit tests for each feature (axes, criteria, streams)
• python/test/: Python-level tests for the wrapper
• CI scripts under ci/ and .github/workflows: matrix builds across compilers, CUDA versions, verification of devcontainers
F. Dev Environment
• .devcontainer/: Docker-based dev images for various CUDA + compiler combos, entrypoint scripts, launch scripts, auto-verification
Relationships and Interactions
• The Build System (CMake) configures and compiles:
• Core Library → static or shared lib
• CLI Executable linked against Core Library
• Examples linked against Core Library
• Python Extension (py_nvbench) built via CMake + pybind11 → links to Core Library
• Tests built and registered with CTest
• At runtime:
• CLI Executable loads GPU drivers, enumerates devices via CUDA Runtime & NVML
• Core Library orchestrates kernel launches, times them with either CUDA Events or CUPTI, collects statistics
• Printers output through selected plugins (markdown, CSV, JSON)
• Python front end invokes the same C++ core under the hood
• Post-processing scripts (scripts/) read JSON or CSV outputs to generate histograms, comparisons
• CI/CD pipelines spin up devcontainers → run CMake configure + build + test across multiple CUDA/toolchain combos → publish artifacts or wheel
Architectural Patterns and Design Principles
• Modular CMake structure: separation of dependency detection, install rules, test registration
• Plugin pattern for output serializers (printer_base, printer_multiplex)
• Axis abstraction for parameter sweeps (static vs runtime)
• Single-responsibility: state management vs kernel launching vs printing vs parsing
• Python binding via pybind11 encapsulating core in a thin layer
Detailed Capture
• Don’t omit submodules: show nvbench/detail and nvbench/internal as child nodes under Core Library
• Python/src and python/cuda as distinct layers of the Python binding
• How .devcontainer images map to matrix builds in CI
• Tests as a separate consumer of the Core Library via CTest