How airflow-helm/charts works
Project Type and Purpose
This is a Helm Chart project specifically designed to deploy Apache Airflow on Kubernetes clusters. The project includes both the main Airflow chart and supporting container images (pgbouncer and postgresql).
Main Components
A. Core Airflow Components:
• Webserver (Web UI)
• Scheduler
• Workers (CeleryExecutor)
• Triggerer
• Flower (Celery monitoring)
B. Database Components:
• PostgreSQL Database
• PgBouncer (Connection Pooler)
• Redis (for Celery)
C. Kubernetes Resources:
• Persistent Volumes (for DAGs and Logs)
• ConfigMaps and Secrets
• Services
• Ingress Controllers
• RBAC Components
[External Access Layer]
[Kubernetes Services Layer]
[Airflow Components Layer]
[Data Storage Layer]
Specific Elements
a) External Access Layer:
• Ingress Controllers (for Webserver and Flower)
• Load Balancers
b) Kubernetes Services Layer:
• Service objects for each component
• PodDisruptionBudgets
• HorizontalPodAutoscalers
c) Airflow Components Layer:
• Webserver Deployment
• Scheduler Deployment
• Worker StatefulSet
• Triggerer Deployment
• Flower Deployment
• PgBouncer Deployment
d) Data Storage Layer:
• PostgreSQL Database
• Redis
• Persistent Volumes for DAGs
• Persistent Volumes for Logs
Connection Types
• Volume mounts (thick lines)
Additional Details
Annotations for:
• Sync Jobs (connections, pools, variables, users)
• Database migration jobs
• Monitoring endpoints (Prometheus)
• Security components (RBAC, ServiceAccounts)