# nvidia/kai-scheduler architecture

> Architecture diagram of the GitHub repository nvidia/kai-scheduler, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia/kai-scheduler
- Repository: https://github.com/nvidia/kai-scheduler

## Overview

##### Project Type and Purpose
- KAI Scheduler is a Kubernetes scheduler extension (“full-stack” in the context of K8s controllers) focused on GPU resource allocation for AI/ML workloads.
- It comprises multiple controllers and webhooks (admission, binder, queuecontroller, podgroupcontroller, podgrouper, nodescaleadjuster, resource-reservation), plus the core scheduler binary/plugin and an operator to manage CRDs and deployments.

##### File Structure
- Top-level “cmd/…” directories each correspond to a separate K8s component executable (e.g., cmd/scheduler, cmd/admission, cmd/binder, cmd/queuecontroller, cmd/podgroupcontroller, cmd/podgrouper, cmd/nodescaleadjuster, cmd/resourcereservation, cmd/operator).
- “pkg/…” contains the internal libraries for each controller, plugin, API clients, informers, caches, actions, frameworks, and admission webhooks.
- “deployments/kai-scheduler” holds Helm chart templates orchestrating all components plus CRDs and RBAC.
- README, Makefile, Dockerfile, and CI workflows indicate build and deployment pipelines.

##### README Insights
- Purpose: GPU-aware scheduling with features like batch scheduling, fairness, elastic workloads, dynamic resource allocation, GPU sharing.
- Installation: Helm deploys an Operator which applies CRDs and manages sub-components.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Control-Plane Components"
        APIS["API Server & etcd"]:::external
        OW["KAI Operator"]:::controlPlane
        AW["Admission Webhook Service"]:::controlPlane
        SCH["Scheduler"]:::controlPlane
    end

    subgraph "Reconciliation Controllers"
        QC["QueueController"]:::reconcile
        PGC["PodGroupController"]:::reconcile
        PGPR["PodGrouper"]:::reconcile
        NSA["NodeScaleAdjuster"]:::reconcile
    end

    subgraph "Binding & Admission"
        BIND["Binder Service"]:::bind
        RR["ResourceReservation Controller"]:::bind
    end

    subgraph "External Services"
        HELM["Helm"]:::external
        GPUO["GPU Operator"]:::external
        METRICS["Prometheus/Grafana"]:::external
        CAS["Cloud Auto-Scalers"]:::external
    end

    Kubectl["kubectl apply"]:::external --> APIS
    APIS --> AW
    AW --> APIS

    APIS --> SCH
    subgraph "Scheduler Internal"
        SC["Cache"]:::controlPlane
        FW["Framework Plugins"]:::controlPlane
        ACT["Actions"]:::controlPlane
        BRG["BindRequest Generator"]:::controlPlane
        SC --> FW --> ACT --> BRG
    end
    SCH -->|"watch Pods/Nodes/Queues/PodGroups"| SC
    BRG -->|"BindRequests"| BIND
    BIND --> APIS

    PGC -->|"update CRD status"| APIS
    QC -->|"update CRD status"| APIS
    APIS --> RR
    RR -->|"patch events"| APIS
    APIS --> NSA
    NSA --> APIS
    APIS --> PGPR
    PGPR --> APIS

    OW -->|"watches KaiScheduler CR"| HELM
    HELM --> OW
    OW --> AW
    OW --> BIND
    OW --> QC
    OW --> PGC
    OW --> PGPR
    OW --> NSA
    OW --> RR
    OW --> SCH

    METRICS -.-> OW
    METRICS -.-> SCH
    GPUO -.-> OW
    CAS -.-> NSA

    click AW "https://github.com/nvidia/kai-scheduler/blob/main/cmd/admission/main.go"
    click BIND "https://github.com/nvidia/kai-scheduler/blob/main/cmd/binder/main.go"
    click QC "https://github.com/nvidia/kai-scheduler/blob/main/cmd/queuecontroller/main.go"
    click PGC "https://github.com/nvidia/kai-scheduler/blob/main/cmd/podgroupcontroller/main.go"
    click PGPR "https://github.com/nvidia/kai-scheduler/blob/main/cmd/podgrouper/main.go"
    click NSA "https://github.com/nvidia/kai-scheduler/blob/main/cmd/nodescaleadjuster/main.go"
    click RR "https://github.com/nvidia/kai-scheduler/blob/main/cmd/resourcereservation/main.go"
    click SCH "https://github.com/nvidia/kai-scheduler/blob/main/cmd/scheduler/main.go"
    click OW "https://github.com/nvidia/kai-scheduler/blob/main/cmd/operator/main.go"
    click HELM "https://github.com/nvidia/kai-scheduler/blob/main/deployments/kai-scheduler/Chart.yaml"

    classDef controlPlane fill:#cce5ff,stroke:#0366d6,color:#0366d6
    classDef reconcile fill:#d4edda,stroke:#28a745,color:#28a745
    classDef bind fill:#fff3cd,stroke:#856404,color:#856404
    classDef external fill:#f8f9fa,stroke:#6c757d,color:#6c757d
```
