How martinohmann/home-ops works
Project type and purpose
• This is an Infrastructure as Code (IaC) & GitOps monorepo for a home Kubernetes environment.
• Purpose:
• Provision Proxmox VMs via Terraform
• Bootstrap and configure k3s clusters (main & storage) via Ansible
• Drive cluster state through Flux (GitOps)
• Automate dependency updates via Renovate and CI via GitHub Actions
File structure
• Top-level directories:
• terraform/ – Terraform modules for Proxmox, Cloudflare, MinIO, etc.
• ansible/ – Ansible playbooks for Kubernetes, Proxmox, WireGuard
• kubernetes/ – Flux-driven Kubernetes manifests, separated into main and storage clusters, each with bootstrap/, components/, apps/, flux/
• .github/ – GitHub Actions workflows, Renovate configuration
• Patterns:
• Two clusters (main & storage) each with identical structure: bootstrap procedures, reusable components, Flux core config, and apps
• Flux Kustomizations and HelmReleases under apps/... for each application
• CI/CD: GitHub Actions in .github/workflows, Renovate config in .github/renovate
README for additional insights
• High-level architecture: Terraform → Proxmox VMs → Ansible → k3s clusters
• Core cluster services: cilium, cert-manager, ingress-nginx, external-dns, longhorn, system-upgrade-controller, volsync, actions-runner
• Flux scans kubernetes/<cluster>/apps recursively for kustomizations to apply
a. Main components:
1. GitHub Monorepo
– Code: terraform/, ansible/, kubernetes/, .github workflows, Renovate
2. CI/CD Layer
– GitHub Actions (e.g., release.yaml, flux-diff.yaml)
– RenovateBot
3. Terraform
– Proxmox provider modules → creates Proxmox VMs
– Cloudflare, MinIO, Authentik, etc. modules
4. Ansible
– k3s cluster provisioning on those VMs
– WireGuard, Proxmox node setup
5. GitOps Engine (Flux)
– Watches kubernetes/main and kubernetes/storage directories
– Applies bootstrap → components → apps via Kustomize & HelmRelease
6. k3s Clusters
– Main cluster: hosts core services + workloads
– Storage cluster: dedicated storage controllers + storage-oriented apps
7. Cluster Services
– Networking: Cilium, Ingress-NGINX
– Certificates: cert-manager
– DNS: external-dns (Cloudflare)
– Storage: Longhorn, Volsync, NFS server (separate host)
– CI runners: actions-runner-controller
– Observability: Prometheus stack, Grafana, Loki, exporters
– Secrets: SOPS, External-Secrets
8. External Services
– DNS provider (Cloudflare)
– Proxmox VE cluster management
– NFS/Bulk storage server
b. Relationships & interactions:
• Developer pushes to GitHub → triggers GitHub Actions & Renovate
• Terraform workflow provisions VMs in Proxmox
• Ansible playbooks configure VMs with k3s, networking, storage prerequisites
• Flux controllers in each cluster watch Git repo → pull manifests → apply via Kustomize/Helm
• Applications (HelmRelease) deployed into namespaces
• Secrets encrypted with SOPS, decrypted by Flux/Ansible/Terraform
• Volsync replicates PersistentVolumes between main & storage clusters
c. Architectural patterns/design principles:
– GitOps: Flux as the single source of truth for cluster state
– Immutable infrastructure: declarative Terraform & Ansible
– Modular IaC: Terraform modules, Ansible roles, reusable Kustomize components
– Separation of concerns: distinct layers (provisioning vs configuration vs application)
– DRY + Reuse: shared components under kubernetes/*/components
d. Key technologies:
– IaC: Terraform (HCL), Ansible (YAML)
– Kubernetes: k3s, Kustomize, HelmRelease
– GitOps: Flux v2
– CI/CD: GitHub Actions, Renovate
– Networking: Cilium, ingress-nginx
– Storage: Longhorn, volsync, NFS, MinIO, CloudNative-PG
– Secrets: Mozilla SOPS, External-Secrets
– Observability: Prometheus, Grafana, Loki, Alertmanager, exporters