How kaito-project/kaito works
Project type and purpose
• This is a Kubernetes operator (written in Go) called “Kaito” that automates the provisioning of GPU nodes and the deployment of AI/ML inference or tuning workloads in a Kubernetes cluster.
• It defines two primary custom resources (CRDs): Workspace and RagEngine (RAG). Controllers reconcile those CRs to create Karpenter/Machine objects, Azure GPU nodes, and then inference/tuning workloads (Deployments, StatefulSets, Jobs).
• It ships with Helm charts for installation (charts/kaito/...), Terraform scripts for infra provisioning, Docker images for each component, and Python-based “presets” for model inference/tuning.
File structure
• Top-level directories:
• api/ – CRD type definitions (v1alpha1, v1beta1).
• pkg/ragengine/ and pkg/workspace/ – Go controllers, webhooks, reconcilers.
• cmd/ – main binaries (ragengine, workspace).
• charts/ – Helm charts for deploying the operator and Demo UI.
• config/ – base Kustomize manifests, RBAC, CRDs.
• docker/ – Dockerfiles for adapter, dataset, ragengine, workspace images.
• presets/ – Python code for inference/tuning presets (vector stores, embeddings, chat templates).
• terraform/ – Terraform modules for provisioning Azure & Kubernetes infra.
• demo/, examples/, docs/, test/ – samples, documentation, e2e tests.
• Patterns:
• Kubernetes operator pattern (CRD + controllers).
• Separation of concerns: operator control-plane vs. workload-plane.
• Infrastructure as Code: Helm + Kustomize + Terraform.
• Presets extensibility: Python modules for custom model logic.
README for insights
• Confirms CRD/controller architecture.
• Describes two main controllers: Workspace controller and external GPU-provisioner (Karpenter).