How a7d-corp/homelab-clusters-fleet works
Project Type and Purpose
• This is an infrastructure management repository for provisioning and managing Kubernetes clusters in a homelab environment.
• It uses FluxCD as the GitOps operator to automate deployments and cluster state reconciliation, coordinating with ClusterAPI/Sidero for cluster creation, Talos for Kubernetes distribution, and Proxmox as the underlying virtualization platform.
• The repository also manages a range of add-ons and base components (e.g., Dex for identity management, OAuth2 Proxy, Pinniped, cert-manager, ingress-nginx, etc.) that are applied across clusters.
File Structure
• Notice the high-level “flux” directory which is subdivided into multiple layers:
• flux/apps: Contains definitions of applications (like dex, oauth2-proxy, and id providers) deployed across clusters.
• flux/clusters: Manages the bootstrapping and lifecycle configuration files for clusters (both bootstrap and specific room101 clusters).
• flux/configs: Provides base configurations and RBAC settings.
• flux/kubernetes: Contains lower-level cluster and patch configurations for Kubernetes resources.
• flux/workloads: Includes add-ons (cert-manager, external-dns, ingress-nginx, etc.), controllers (cluster-api-operator and others), and early bootstrap components.
• Look at the “.github” directory which holds workflows for CI/CD (automated changelog and release PR creation) and the “hack” folder that includes scripts for operational support like credential encryption and VM management.
• Multiple kustomization.yaml files signal a use of Kustomize for applying overlays and managing environment-specific customizations.
README File
• The README confirms that the repository is used by FluxCD to manage clusters via ClusterAPI/Sidero running on Proxmox, giving you a clear idea that it is chiefly focused on cluster management and GitOps-driven operations.
• The docs directory further supports this by detailing topics like networking, image factory, and specific notes on bootstrap mechanisms.
a. Main Components:
• GitOps Operator (FluxCD): The heart of the system, orchestrating state synchronization from the repository to the Kubernetes clusters.
• Cluster Management: This includes clusters bootstrapping configurations, cluster nodes, and underlying Talos configurations along with ClusterAPI/Sidero integrations.
• Application and Add-on Deployment: This area covers core services like Dex, OAuth2 Proxy, and Pinniped along with other add-ons (ingress controllers, cert-manager, external-dns, etc.).
• Infrastructure Underlay: External services like Proxmox (virtualization layer) and other external systems (e.g., Sidero integrations or cloud controllers).
• CI/CD Pipelines: Represented by the GitHub workflows that automate releases and validate changelogs.
• Operational Scripts and Tools: The “hack” directory which holds custom scripts for credential management, VM lifecycle operations, and certificate generation.
b. Relationships and Interactions:
• FluxCD continuously reconciles the state from the Git repository to the clusters, pulling in specifics from directories such as flux/apps and flux/configs.
• Clusters are provisioned via bootstrapping in flux/clusters with a direct connection to the virtualization layer (Proxmox) via Talos.
• The add-ons and controllers under flux/workloads are applied to each cluster, ensuring that functions like identity management, networking, and ingress are available.
• CI/CD pipelines interact with Git through automation workflows in the .github directory.
c. Key Architectural Patterns and Design Principles:
• GitOps: A declarative approach to cluster configuration management where the Git repo is the single source of truth.
• Layered Configuration Management: Use of Kustomize to manage and overlay configurations for different targets (base vs. environment-specific).
• Modular and Reusable Components: Clear separation between base apps, add-ons, bootstrap configurations, and patch-level customization allows for scaling and modularity.
d. Technologies and Frameworks:
• FluxCD for GitOps operations.
• ClusterAPI/Sidero for Kubernetes cluster lifecycle management.
• Talos for lightweight Kubernetes operating systems.
• Proxmox as the virtualization backend.
• Kustomize for configuration overlays.
• Various Helm charts (seen in helmrelease and helmrepository YAML files) for managing application deployments.
• CI/CD tooling through GitHub Actions.
Detail Orientation
• Annotations or side notes for special areas like the add-ons (cert-manager, ingress-nginx) and controllers which have been segregated in their own directories.
• It’s important to note the dual roles: one part of the repo is managing cluster lifecycle and bootstrap, and another part is continuously ensuring that applications and policies are consistently applied across the clusters.