How aifoundry-org/oxide-controller works
Project Type and Purpose
• oxide-controller is a Go-based control plane service (a backend “controller”) that automates lifecycle management of a Kubernetes cluster on an Oxide sled.
• It exposes a small REST API for scaling worker nodes and continuously reconciles cluster state.
• It can be run as a standalone CLI tool or deployed into the very cluster it manages (via Helm).
File Structure Analysis
• cmd/ – Entry points (CLI commands: main.go, build.go, root.go).
• pkg/cluster/ – Core cluster-management logic (provisioning, SSH, Helm orchestration, health checks).
• pkg/server/ – REST API server implementation.
• pkg/image, pkg/util, pkg/log, pkg/oxide – Helpers (image builds, file/SSH utilities, logging, profile management).
• chart/ & manifests/ – Helm chart and Kubernetes manifests for in-cluster deployment.
• Dockerfile, Makefile, CI workflows – Build, test, and publish pipelines.
Key Architectural Patterns
• Reconciliation Loop – Continuously inspect cluster state and self-heal (common in Kubernetes controllers).
• Separation of Concerns – CLI vs. server process vs. cluster-provisioning library.
• Declarative Deployment – Helm chart and Kubernetes manifests for “operator runs inside cluster.”
Main Components
a. CLI Client (oxide-controller CLI)
b. Controller Service (REST API + reconciliation loop)
c. Kubernetes Cluster
• Control Plane (initial node + kube-apiserver)
• Worker Nodes
d. Helm & Tiller (Helm v2/v3 client components invoked by controller)
e. OCI Registry (hosts controller images)
f. SSH Layer (key-pair generation, node bootstrap)
g. kubeconfig Secret & Kubernetes RBAC objects
h. Optional External Service: Tailscale
Relationships and Data Flow
• CLI → SSH → Bootstrap Control Plane → retrieve join token + kubeconfig
• Controller → Kubernetes API Server (via in-cluster ServiceAccount or kubeconfig)
• Controller → Helm API to deploy itself & create RBAC resources
• REST API calls (e.g., GET/POST /nodes/workers) → Controller → Kubernetes API → adjust worker Deployment/StatefulSet
• Optional Tailscale installation via Controller → SSH → Nodes
Technologies & Frameworks
• Go modules, client-go, helm Go SDK
• Docker, OCI image registry
• Helm chart templating
• Kubernetes RBAC, ServiceAccount, StatefulSet
• SSH (golang.org/x/crypto/ssh)
• Tailscale (optional sidecar/install script)