How nvidia/kai-scheduler works
Project Type and Purpose
• KAI Scheduler is a Kubernetes scheduler extension (“full-stack” in the context of K8s controllers) focused on GPU resource allocation for AI/ML workloads.
• It comprises multiple controllers and webhooks (admission, binder, queuecontroller, podgroupcontroller, podgrouper, nodescaleadjuster, resource-reservation), plus the core scheduler binary/plugin and an operator to manage CRDs and deployments.
File Structure
• Top-level “cmd/…” directories each correspond to a separate K8s component executable (e.g., cmd/scheduler, cmd/admission, cmd/binder, cmd/queuecontroller, cmd/podgroupcontroller, cmd/podgrouper, cmd/nodescaleadjuster, cmd/resourcereservation, cmd/operator).
• “pkg/…” contains the internal libraries for each controller, plugin, API clients, informers, caches, actions, frameworks, and admission webhooks.
• “deployments/kai-scheduler” holds Helm chart templates orchestrating all components plus CRDs and RBAC.
• README, Makefile, Dockerfile, and CI workflows indicate build and deployment pipelines.
README Insights
• Purpose: GPU-aware scheduling with features like batch scheduling, fairness, elastic workloads, dynamic resource allocation, GPU sharing.
• Installation: Helm deploys an Operator which applies CRDs and manages sub-components.