How crossplane-contrib/provider-aws works
Project type and purpose
• This is a Kubernetes “infrastructure provider” plugin (provider-aws) for the Crossplane control plane.
• It defines Custom Resource Definitions (CRDs) for dozens of AWS services, implements controllers that reconcile those CRDs, and uses AWS SDK calls to provision real AWS resources.
• It also includes code‐generation pipelines, example manifests, packaging configuration (Helm/OCI), and CI/CD workflows.
File structure
• Top‐level directories:
• apis – Go types and generated deep‐copy, managed‐resource, resolver code defining CRDs and schema for each AWS service.
• pkg/clients – AWS client wrappers and fakes.
• pkg/controller – Individual controller setup, reconcile logic and conversion for each CRD.
• cmd/provider – The main binary that starts the controller manager.
• package/crds – Packaged YAML CRD manifests for installation.
• examples – Example Kubernetes YAMLs showing how users create CRs.
• cluster – Local and image build scripts for running integration tests or deploying on a cluster.
• templates & CODE_GENERATION.md – Code‐gen templates and instructions for adding new APIs.
• .github, Makefile, workflows – CI/CD pipelines, linting, release automation.
README for additional insights
• Confirms that this repo produces:
• CRDs modeling AWS infrastructure
• Controllers that provision AWS resources
• Crossplane portable abstractions implementations
• References a code‐generation pipeline and crossplane.io docs for installation/deployment.
a. Main components
• Kubernetes API Server (+ CustomResourceDefinitions)
• provider-aws Controller Manager (a Go binary running controller‐runtime)
– api schemata (types from apis/)
– controllers (pkg/controller/…)
– AWS client layer (pkg/clients)
• AWS Public APIs (one per AWS service domain)
• Crossplane Core (separate control‐plane that delegates to providers)
• Code Generation Toolchain (generate.go, templates/)
• CI/CD pipelines (GitHub workflows, Makefile)
• User Workstation (kubectl + crossplane CLI applying example CRs)
b. Relationships and interactions
• User → kubectl: applies CR YAML from examples/ or generated CRDs.
• API Server → CRDs stored in etcd; notifies controller manager via informers.
• Controller Manager
– Watches CRs → reads desired state
– Calls pkg/clients → AWS SDK calls to create/update/delete AWS resources
– Updates CR status/conditions back into Kubernetes
• Crossplane Core → may call provider-aws as an “Extension” via the same API+controllers.
• Code‐Gen Toolchain → reads generator-config.yaml in apis/*, outputs new types/controllers.
• CI/CD Workflow → builds container images, runs tests, pushes CRD YAML to package/.
c. Architectural patterns and design principles
• Kubernetes Operator pattern (watch → reconcile loop).
• Separation of concerns:
– api types (apis/)
– control logic (pkg/controller/)
– infrastructure client abstraction (pkg/clients/)
• Code generation for boilerplate & consistent APIs.
• Test-driven: each client and controller has unit tests and integration test scripts in cluster/local.
• GitHub Actions for CI, keeping release cadence.
d. Key technologies, frameworks, libraries
• Go + controller-runtime
• Kubernetes API machinery (client-go, code-generator)
• AWS Go SDK v2 (wrapped in pkg/clients)
• GitHub Actions, Makefile
• Crossplane’s plugin and code‐generation pipelines
Be detailed and capture essential elements
• Break out controllers by service group if zoomed in (e.g., apis/ec2 + pkg/controller/ec2).
• Common client abstraction layer under all controllers.
• Code‐generation pipeline as a separate flow feeding into apis/ and pkg/controller/.
• CI/CD workflows building and pushing container images and CRD packages.