# kaito-project/kaito architecture

> Architecture diagram of the GitHub repository kaito-project/kaito, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kaito-project/kaito
- Repository: https://github.com/kaito-project/kaito

## Overview

##### Project type and purpose
- This is a Kubernetes operator (written in Go) called “Kaito” that automates the provisioning of GPU nodes and the deployment of AI/ML inference or tuning workloads in a Kubernetes cluster.
- It defines two primary custom resources (CRDs): Workspace and RagEngine (RAG). Controllers reconcile those CRs to create Karpenter/Machine objects, Azure GPU nodes, and then inference/tuning workloads (Deployments, StatefulSets, Jobs).
- It ships with Helm charts for installation (`charts/kaito/...`), Terraform scripts for infra provisioning, Docker images for each component, and Python-based “presets” for model inference/tuning.

##### File structure
- Top-level directories:
- `api/` – CRD type definitions (v1alpha1, v1beta1).
- `pkg/ragengine/` and `pkg/workspace/` – Go controllers, webhooks, reconcilers.
- `cmd/` – main binaries (ragengine, workspace).
- `charts/` – Helm charts for deploying the operator and Demo UI.
- `config/` – base Kustomize manifests, RBAC, CRDs.
- `docker/` – Dockerfiles for adapter, dataset, ragengine, workspace images.
- `presets/` – Python code for inference/tuning presets (vector stores, embeddings, chat templates).
- `terraform/` – Terraform modules for provisioning Azure & Kubernetes infra.
- `demo/`, `examples/`, `docs/`, `test/` – samples, documentation, e2e tests.
- Patterns:
- Kubernetes operator pattern (CRD + controllers).
- Separation of concerns: operator control-plane vs. workload-plane.
- Infrastructure as Code: Helm + Kustomize + Terraform.
- Presets extensibility: Python modules for custom model logic.

##### README for insights
- Confirms CRD/controller architecture.
- Describes two main controllers: Workspace controller and external GPU-provisioner (Karpenter).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Control Plane
    subgraph "Control Plane"
        direction TB
        User["User / CLI (kubectl)"]:::external
        APIServer["Kubernetes API Server"]:::controlPlane
        subgraph "CRD Definitions"
            direction TB
            WorkspaceCRD["Workspace CRD"]:::controlPlane
            RagEngineCRD["RagEngine CRD"]:::controlPlane
        end
        WorkspaceController["Workspace Controller"]:::controlPlane
        RAGEngineController["RAGEngine Controller"]:::controlPlane
        SKUHandlers["GPU SKU Handlers"]:::controlPlane
        K8sClient["K8s Client & Utils"]:::controlPlane
        PresetsWS["Workspace Presets"]:::controlPlane
        PresetsRAG["RAGEngine Presets"]:::controlPlane
    end

    %% External Provisioner
    subgraph "External Provisioner"
        direction TB
        Karpenter["Karpenter Controller"]:::external
        AzureRM["Azure Resource Manager"]:::external
    end

    %% Worker Plane
    subgraph "Worker Plane"
        direction TB
        Nodes["Provisioned GPU Nodes"]:::worker
        Scheduler["Kubernetes Scheduler & Kubelet"]:::worker
        subgraph "Inference/Tuning Workloads"
            direction TB
            Pods["Inference/Tuning Pods"]:::worker
            VectorStore["Vector Store (Faiss)"]:::worker
        end
    end

    %% Installation & Infra Tools
    subgraph "Installation & Infra"
        direction TB
        Helm["Helm Charts"]:::infra
        Kustomize["Kustomize & RBAC Manifests"]:::infra
        Terraform["Terraform Modules"]:::infra
        ContainerRegistry["Container Registry"]:::infra
    end

    %% Relationships
    User -->|"kubectl apply CRs"| APIServer
    APIServer -->|watch| WorkspaceController
    APIServer -->|watch| RAGEngineController
    WorkspaceController -->|creates| WorkspaceCRD
    RAGEngineController -->|creates| RagEngineCRD
    WorkspaceController -->|calls| SKUHandlers
    RAGEngineController -->|calls| SKUHandlers
    SKUHandlers -->|Machine CRs| Karpenter
    Karpenter -->|calls| AzureRM
    WorkspaceController -->|deploys| Pods
    RAGEngineController -->|deploys| Pods
    PresetsWS -->|plugins into| WorkspaceController
    PresetsRAG -->|plugins into| RAGEngineController
    Pods -->|store/retrieve| VectorStore
    Pods -->|scheduled by| Scheduler
    Scheduler -->|runs on| Nodes
    ContainerRegistry -->|hosts images for| WorkspaceController
    ContainerRegistry -->|hosts images for| RAGEngineController
    ContainerRegistry -->|hosts images for| Pods
    Helm -->|installs| WorkspaceController
    Helm -->|installs| RAGEngineController
    Kustomize -->|applies| APIServer
    Terraform -->|provisions| Nodes
    Terraform -->|configures| AzureRM

    %% Click Events
    click WorkspaceController "https://github.com/kaito-project/kaito/blob/main/cmd/workspace/main.go"
    click WorkspaceCRD "https://github.com/kaito-project/kaito/blob/main/api/v1alpha1/workspace_types.go"
    click RagEngineCRD "https://github.com/kaito-project/kaito/blob/main/api/v1alpha1/ragengine_types.go"
    click RAGEngineController "https://github.com/kaito-project/kaito/blob/main/cmd/ragengine/main.go"
    click WorkspaceCRD "https://github.com/kaito-project/kaito/blob/main/api/v1beta1/workspace_types.go"
    click RagEngineCRD "https://github.com/kaito-project/kaito/blob/main/api/v1beta1/ragengine_types.go"
    click Helm "https://github.com/kaito-project/kaito/tree/main/charts/kaito/workspace"
    click Helm "https://github.com/kaito-project/kaito/tree/main/charts/kaito/ragengine"
    click Helm "https://github.com/kaito-project/kaito/tree/main/charts/DemoUI/inference"
    click Kustomize "https://github.com/kaito-project/kaito/blob/main/config/crd/bases/kaito.sh_workspaces.yaml"
    click Kustomize "https://github.com/kaito-project/kaito/blob/main/config/crd/bases/kaito.sh_ragengines.yaml"
    click Kustomize "https://github.com/kaito-project/kaito/blob/main/config/manager/manager.yaml"
    click Terraform "https://github.com/kaito-project/kaito/blob/main/terraform/main.tf"
    click Terraform "https://github.com/kaito-project/kaito/blob/main/terraform/kubernetes.tf"
    click Terraform "https://github.com/kaito-project/kaito/blob/main/terraform/gpu-provisioner-values.tmpl"
    click Terraform "https://github.com/kaito-project/kaito/blob/main/terraform/variables.tf"
    click PresetsWS "https://github.com/kaito-project/kaito/tree/main/presets/workspace/inference/chat_templates"
    click PresetsWS "https://github.com/kaito-project/kaito/tree/main/presets/workspace/inference/*"
    click PresetsWS "https://github.com/kaito-project/kaito/tree/main/presets/workspace/tuning/*"
    click PresetsWS "https://github.com/kaito-project/kaito/tree/main/presets/workspace/models/*"
    click PresetsRAG "https://github.com/kaito-project/kaito/blob/main/presets/ragengine/main.py"
    click PresetsRAG "https://github.com/kaito-project/kaito/blob/main/presets/ragengine/inference/inference.py"
    click PresetsRAG "https://github.com/kaito-project/kaito/blob/main/presets/ragengine/vector_store/faiss_store.py"
    click PresetsRAG "https://github.com/kaito-project/kaito/tree/main/presets/ragengine/embedding/*"
    click SKUHandlers "https://github.com/kaito-project/kaito/blob/main/pkg/sku/azure_sku_handler.go"
    click SKUHandlers "https://github.com/kaito-project/kaito/blob/main/pkg/sku/cloud_sku_handler.go"
    click K8sClient "https://github.com/kaito-project/kaito/blob/main/pkg/k8sclient/client.go"
    click K8sClient "https://github.com/kaito-project/kaito/blob/main/pkg/utils/plugin/plugin.go"

    %% Styles
    classDef controlPlane fill:#fdeffe,stroke:#d6336c;
    classDef external fill:#e8f7ff,stroke:#339af0;
    classDef worker fill:#e6ffed,stroke:#2f9e44;
    classDef infra fill:#fff7e6,stroke:#fd7e14;
    classDef default fill:#ffffff,stroke:#333333;
```
