How aws/eks-anywhere works
Project type and purpose
• Amazon EKS Anywhere is a Go-based CLI plus Kubernetes operator (“controller-manager”) that enables on-premises Kubernetes cluster lifecycle management.
• Key roles: the user CLI (eksctl-anywhere & eks-a-tool), CRD definitions, controller manager logic (reconciliation loops), provider plugins (CloudStack, Nutanix, Snow, vSphere, Docker, Tinkerbell), and auxiliary components (OIDC server, GitOps integration via Flux).
High-level structure
• Top-level “cmd/…” directories hold CLI entry points.
• “config/…” holds Kustomize manifests for CRDs, RBAC, and controller deployment.
• “controllers/…” contains Kubernetes controller implementations.
• “pkg/…” houses core libraries: API object definitions, provider implementations, cluster-API glue, bootstrap logic, workflows, utilities.
• “internal/…” holds private modules (AWS SDK, OIDC server).
• “manager/…” builds the controller-manager image.
• “config/crd” and “config/default” → CRD YAML and default patches.
• “cmd/integration_test” → end-to-end test harness.
Main components
A. User CLI Layer
• eksctl-anywhere & eks-a-tool binaries
• Parses user flags, builds ClusterConfig objects (pkg/api → pkg/cluster), writes CRDs to management cluster.
B. Kubernetes Control Plane (Management Cluster)
• API Server + etcd
• Custom Resource Definitions (CRDs) for Cluster, MachineConfigs, DataCenterConfigs, FluxConfig, OIDCConfig, Bundles
• Admission Webhooks (config/webhook)
C. Controller-Manager Deployment
• Controllers: ClusterController, MachineDeploymentController, NodeUpgradeController, ControlPlaneUpgradeController, etc.
• Provider Reconcilers (cloudstack, nutanix, snow, vsphere, docker, tinkerbell)
• Bootstrapper & Conformance runner (pkg/bootstrapper, pkg/conformance)
• GitOps integrator (pkg/gitops/flux)
• OIDC server (internal/pkg/oidc)
D. External Provider APIs
• CloudStack REST API, Nutanix Prism API, AWS Snow API, vSphere vCenter, Docker�d, Tinkerbell services
• AWS IAM for EKS IAMAuthenticator (pkg/awsiam)
E. Supporting Services & Repos
• Container registries (docker, ECR)
• Git repos for Flux
• S3 buckets (pkg/s3)
• Bundle storage (bundles of EKS-D, etc.)
Interaction flow
1. Operator runs CLI → constructs Cluster CR & associated objects → writes them to the management cluster API server over kubeconfig.
2. Controllers watch CRs → reconciliation loops pick up new/changed CRs.
3. Provider plugins perform API calls to provision VMs, networks, load balancers, bootstrap Kubernetes on nodes.
4. Bootstrapper injects core EKS-D components (etcd, kube-apiserver, kubelet).
5. Flux GitOps deploys add-ons if requested (pkg/gitops/flux).
6. For upgrades/conformance: node-upgrader controllers or conformance controllers execute in pods.
7. OIDC server may serve discovery/keys for cluster identity.
Architectural patterns & principles
• Kubernetes Operator pattern: declarative CRDs + controller loops.
• Plugin-based provider abstraction: pkg/providers and pkg/clusterapi glue enabling swap-in of multiple infrastructures.
• Separation of concerns: CLI vs. controllers vs. provider modules.
• GitOps extension via Flux integration.
• Use of build scripts and Kustomize for packaging and deployment.