How pulumi/pulumi-kubernetes-operator works
Project Type & Purpose
• This repository implements an open‐source Kubernetes Operator––the Pulumi Kubernetes Operator––plus a companion Agent service.
• Purpose: Expose Pulumi Stacks as first‐class Kubernetes Custom Resources, enabling a GitOps‐style CI/CD workflow inside any Kubernetes cluster.
File/Package Structure
• Top‐level:
• deploy/helm & deploy/crds → Helm chart and CRD definitions for operator deployment
• operator/ → the core operator code (controllers, webhooks, CRD API types, Kustomize configs, e2e tests)
• agent/ → standalone gRPC service (and CLI) that actually executes Pulumi programs
• examples/, stack-examples/ → sample Stack CRs and Pulumi programs
• docs/ → architecture/design, metrics, workflows, troubleshooting
• Patterns:
• Kubernetes Operator pattern: separate controller loops per CRD (Stack, Workspace, Update, Program)
• Microservice split: operator (in‐cluster controller & webhook) vs. agent (gRPC executor)
• Helm/Kustomize for deployment manifests
Key Architectural Components
a. Kubernetes API Server & CustomResourceDefinitions
• CRDs: Stacks, Workspaces, Programs, Updates (deploy/crds & operator/config)
• Operator’s Admission Webhooks for defaulting/validation on Workspace CRs
b. Pulumi Operator Deployment
• manager Deployment (operator/cmd, operator/internal/controller, operator/internal/webhook)
• Controller modules:
• auto controller (Workspace & Update CRs)
• pulumi controller (Stack & Program CRs)
• RBAC, Leader Election, Metrics Service for Prometheus (operator/config/)
c. Agent Service
• agent/cmd & agent/pkg/server → gRPC server that runs Pulumi CLI under the hood
• agent/pkg/client → gRPC client used by controllers to invoke the agent
• Designed to isolate Pulumi executions from the in‐cluster operator process
d. External Integrations
• Pulumi Cloud (state backend, access tokens in Kubernetes Secrets)
• Git repos, Flux sources, custom sources (examples/…)
• Prometheus metrics endpoint scraped from operator manager
Component Interactions & Data Flows
1. User commits Infrastructure as Code or creates a Stack CR (via kubectl/Helm/Pulumi)
2. CR posted to Kubernetes API Server
3. Operator controller (watching CRs) receives the event → reconciliation loop
4. Controller builds a gRPC request → calls Agent Service (in‐cluster or remote)
5. Agent authenticates to Pulumi Cloud / backends → runs pulumi up / pulumi destroy
6. Pulumi providers (AWS/Azure/GCP/K8s) are invoked → resources created/updated
7. Agent streams logs & results back to Operator via gRPC
8. Operator updates CR status & emits Prometheus metrics
9. GitOps feedback loop: Git commits ↔ Kubernetes CRs
Architectural Patterns & Design Principles
• Kubernetes Operator pattern: use controllers, finalizers, status subresources
• Webhook pattern for CR defaulting & validation
• Separation of concerns: operator orchestrates, agent executes (isolation & scalability)
• Declarative infra as data: CRDs represent desired state, controllers converge on it