# nvidia/gpu-operator architecture

> Architecture diagram of the GitHub repository nvidia/gpu-operator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia/gpu-operator
- Repository: https://github.com/nvidia/gpu-operator

## Overview

##### Project Type and Purpose
- This is the NVIDIA GPU Operator, a Kubernetes Operator written in Go.
- Purpose: automate provisioning and lifecycle management of NVIDIA CUDA drivers, device plugins, container runtimes, DCGM-based monitoring, node labelling, and other GPU-related software components on a Kubernetes cluster via custom resources (CRDs).

##### File Structure
- **api/** – Go types and code-generated clients for two CRDs: ClusterPolicy (v1) and NvidiaDriver (v1alpha1).
- **controllers/** – Reconciliation loops (controllers) implementing the Operator business logic (clusterpolicy_controller, nvidiadriver_controller, upgrade_controller, state_manager, etc.).
- **internal/** – Shared libraries: condition computation, rendering Kubernetes manifests, state management logic, validators, utilities.
- **cmd/** – Three CLI binaries:
- gpu-operator (the main Operator binary)
- gpuop-cfg (validates and generates YAML for clusterpolicy & CSV)
- nvidia-validator (preflight/metrics validation)
- **config/** and **deployments/** / **bundle/** / **manifests/** – Kustomize, Helm charts, OLM bundle definitions, and static YAML assets used to install the Operator and its operands.
- **assets/** – Per-component Kubernetes resources (ServiceAccounts, Roles, DaemonSets) for GPU driver installer, device plugin, DCGM exporter, MIG manager, vGPU manager, etc.
- **tests/** – Unit tests, E2E framework, shell scripts, YAML workloads to validate behavior.

##### Extract Key Insights from README
- The Operator leverages the Kubernetes Device Plugin framework and Operator Framework.
- Administrators use a single standard OS image; the Operator deploys runtime components as containers.
- Scenarios: cloud or on-prem clusters that need rapid scaling of GPU nodes and automated software provisioning.

##### Components
a. **Control Plane:**
- Kubernetes API Server & etcd (cluster control plane).
- GPU Operator Pod (controller manager) with watchers for CRDs.
- CustomResourceDefinitions: ClusterPolicy and NvidiaDriver API groups.
b. **Data Plane (Worker Nodes):**
- DaemonSets for:
- NVIDIA Driver installer (privileged init).
- NVIDIA Container Runtime.
- Kubernetes Device Plugin for GPUs.
- DCGM Exporter (monitoring).
- Node Feature Discovery (optional chart).
- State managers (e.g., MIG, VFIO, Sandbox, MPS).
- ConfigMaps & Secrets (per-component configuration).
c. **External Dependencies / Tools:**
- Container registry (NVIDIA images).
- gpuop-cfg & nvidia-validator CLIs (admin-side).
- OLM / Helm (installer).
- Prometheus / Grafana (monitoring integration).

##### Relationships & Data Flows
- Operator Pod <–watches–>­ API Server for CR events (ClusterPolicy / NvidiaDriver).
- Operator Pod uses client-go to create/update child resources (DaemonSets, ConfigMaps, Roles).
- Worker nodes run DaemonSets which pull images from registry, install drivers, expose device plugins.
- DCGM exporter publishes GPU metrics to Prometheus.
- Admin CLIs validate and push CRs to the API Server.

##### Architectural Patterns & Design Principles
- **Operator Pattern:** declarative reconciliation loops.
- **CRD-centric design:** two custom API groups.
- **Modular render library:** templating YAML manifests.
- **Kubernetes-native packaging:** Helm charts, Kustomize, OLM bundle.
- **Test automation:** unit, integration, e2e frameworks.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Control Plane
    subgraph "Control Plane"
        APISrv["API Server"]:::control
        Etcd["etcd"]:::control
        APISrv --> Etcd

        CPCRD["ClusterPolicy CRD"]:::control
        NDCRD["NvidiaDriver CRD"]:::control
        CPCRD -.-> APISrv
        NDCRD -.-> APISrv

        CLIPolicy["gpuop-cfg CLI"]:::external
        CLIVal["nvidia-validator CLI"]:::external
        HelmOLM["OLM/Helm Installer"]:::external

        subgraph "GPU Operator Pod"
            OPBin["Operator Binary"]:::control
            subgraph "Controllers"
                CPCtrl["ClusterPolicy Controller"]:::control
                NDCtrl["NvidiaDriver Controller"]:::control
                UpCtrl["Upgrade Controller"]:::control
                StMgr["State Manager Controller"]:::control
            end
            subgraph "Libraries"
                RendLib["Render Library"]:::library
                StateLib["State Management Library"]:::library
                CondLib["Conditions Library"]:::library
                ValLib["Validation Library"]:::library
            end
        end
    end

    %% Worker Nodes
    subgraph "Worker Nodes"
        DSDriver["Driver Installer DaemonSet"]:::worker
        DSPlugin["Device Plugin DaemonSet"]:::worker
        DSToolkit["Container Toolkit DaemonSet"]:::worker
        DSDCGM["DCGM Exporter DaemonSet"]:::worker
        DSMig["MIG Manager DaemonSet"]:::worker
        DSVfio["VFIO Manager DaemonSet"]:::worker
        DSVgpu["vGPU Manager DaemonSet"]:::worker
        DSMPS["MPS Control DaemonSet"]:::worker
        DSSandbox["Sandbox Device Plugin DaemonSet"]:::worker
        DSNodeStatus["Node Status Exporter DaemonSet"]:::worker
    end

    %% External Systems
    Registry["Container Registry"]:::external
    Prometheus["Prometheus"]:::external

    %% Data Flows
    CLIPolicy --> APISrv
    CLIVal --> APISrv
    HelmOLM --> APISrv
    APISrv -->|"CR Events"| OPBin

    OPBin --> CPCtrl
    OPBin --> NDCtrl
    OPBin --> UpCtrl
    OPBin --> StMgr

    CPCtrl --> RendLib
    CPCtrl --> StateLib
    NDCtrl --> RendLib
    NDCtrl --> StateLib
    StMgr --> StateLib
    UpCtrl --> StateLib

    OPBin --> DSDriver
    OPBin --> DSPlugin
    OPBin --> DSToolkit
    OPBin --> DSDCGM
    OPBin --> DSMig
    OPBin --> DSVfio
    OPBin --> DSVgpu
    OPBin --> DSMPS
    OPBin --> DSSandbox
    OPBin --> DSNodeStatus

    DSDriver -->|"pulls images"| Registry
    DSPlugin -->|"pulls images"| Registry
    DSToolkit -->|"pulls images"| Registry
    DSDCGM -->|"pulls images"| Registry
    DSMig -->|"pulls images"| Registry
    DSVfio -->|"pulls images"| Registry
    DSVgpu -->|"pulls images"| Registry
    DSMPS -->|"pulls images"| Registry
    DSSandbox -->|"pulls images"| Registry
    DSNodeStatus -->|"pulls images"| Registry

    DSDCGM -->|"metrics"| Prometheus

    %% Click Events
    click CPCRD "https://github.com/nvidia/gpu-operator/blob/main/api/nvidia/v1/clusterpolicy_types.go"
    click NDCRD "https://github.com/nvidia/gpu-operator/blob/main/api/nvidia/v1alpha1/nvidiadriver_types.go"
    click CPCtrl "https://github.com/nvidia/gpu-operator/blob/main/controllers/clusterpolicy_controller.go"
    click NDCtrl "https://github.com/nvidia/gpu-operator/blob/main/controllers/nvidiadriver_controller.go"
    click UpCtrl "https://github.com/nvidia/gpu-operator/blob/main/controllers/upgrade_controller.go"
    click StMgr "https://github.com/nvidia/gpu-operator/blob/main/controllers/state_manager.go"
    click OPBin "https://github.com/nvidia/gpu-operator/blob/main/cmd/gpu-operator/main.go"
    click CLIPolicy "https://github.com/nvidia/gpu-operator/blob/main/cmd/gpuop-cfg/main.go"
    click CLIVal "https://github.com/nvidia/gpu-operator/blob/main/cmd/nvidia-validator/main.go"
    click RendLib "https://github.com/nvidia/gpu-operator/blob/main/internal/render/render.go"
    click StateLib "https://github.com/nvidia/gpu-operator/blob/main/internal/state/manager.go"
    click CondLib "https://github.com/nvidia/gpu-operator/blob/main/internal/conditions/conditions.go"
    click ValLib "https://github.com/nvidia/gpu-operator/blob/main/internal/validator/validator.go"
    click DSPlugin "https://github.com/nvidia/gpu-operator/blob/main/assets/state-device-plugin/0500_daemonset.yaml"
    click DSDriver "https://github.com/nvidia/gpu-operator/blob/main/assets/state-driver/0500_daemonset.yaml"
    click DSToolkit "https://github.com/nvidia/gpu-operator/blob/main/assets/state-container-toolkit/0500_daemonset.yaml"
    click DSDCGM "https://github.com/nvidia/gpu-operator/blob/main/assets/state-dcgm-exporter/0800_daemonset.yaml"
    click DSMig "https://github.com/nvidia/gpu-operator/blob/main/assets/state-mig-manager/0600_daemonset.yaml"
    click DSVfio "https://github.com/nvidia/gpu-operator/blob/main/assets/state-vfio-manager/0600_daemonset.yaml"
    click DSVgpu "https://github.com/nvidia/gpu-operator/blob/main/assets/state-vgpu-manager/0500_daemonset.yaml"
    click DSMPS "https://github.com/nvidia/gpu-operator/blob/main/assets/state-mps-control-daemon/0400_daemonset.yaml"
    click DSSandbox "https://github.com/nvidia/gpu-operator/blob/main/assets/state-sandbox-device-plugin/0500_daemonset.yaml"
    click DSNodeStatus "https://github.com/nvidia/gpu-operator/blob/main/assets/state-node-status-exporter/0700_daemonset.yaml"

    %% Styles
    classDef control fill:#AEDFF7,stroke:#036; 
    classDef worker fill:#D7F9D1,stroke:#070;
    classDef external fill:#FFE4B5,stroke:#EA7;
    classDef library fill:#F7E1F7,stroke:#A0A;
```
