# airflow-helm/charts architecture

> Architecture diagram of the GitHub repository airflow-helm/charts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/airflow-helm/charts
- Repository: https://github.com/airflow-helm/charts

## Overview

##### Project Type and Purpose
This is a Helm Chart project specifically designed to deploy Apache Airflow on Kubernetes clusters. The project includes both the main Airflow chart and supporting container images (pgbouncer and postgresql).

##### Main Components

A. Core Airflow Components:
- Webserver (Web UI)
- Scheduler
- Workers (CeleryExecutor)
- Triggerer
- Flower (Celery monitoring)

B. Database Components:
- PostgreSQL Database
- PgBouncer (Connection Pooler)
- Redis (for Celery)

C. Kubernetes Resources:
- Persistent Volumes (for DAGs and Logs)
- ConfigMaps and Secrets
- Services
- Ingress Controllers
- RBAC Components

**[External Access Layer]**

**[Kubernetes Services Layer]**

**[Airflow Components Layer]**

**[Data Storage Layer]**

##### Specific Elements

a) External Access Layer:
- Ingress Controllers (for Webserver and Flower)
- Load Balancers

b) Kubernetes Services Layer:
- Service objects for each component
- PodDisruptionBudgets
- HorizontalPodAutoscalers

c) Airflow Components Layer:
- Webserver Deployment
- Scheduler Deployment
- Worker StatefulSet
- Triggerer Deployment
- Flower Deployment
- PgBouncer Deployment

d) Data Storage Layer:
- PostgreSQL Database
- Redis
- Persistent Volumes for DAGs
- Persistent Volumes for Logs

##### Connection Types

- Volume mounts (thick lines)

##### Additional Details

Annotations for:
- Sync Jobs (connections, pools, variables, users)
- Database migration jobs
- Monitoring endpoints (Prometheus)
- Security components (RBAC, ServiceAccounts)

## Diagram (Mermaid)

```mermaid
graph TB
    %% Define styles
    classDef external fill:#2374f7,stroke:#000,stroke-width:2px,color:#fff
    classDef internal fill:#47d147,stroke:#000,stroke-width:2px
    classDef database fill:#ffd700,stroke:#000,stroke-width:2px
    classDef storage fill:#9933ff,stroke:#000,stroke-width:2px
    classDef monitoring fill:#ff8c1a,stroke:#000,stroke-width:2px
    classDef config fill:#e6e6e6,stroke:#000,stroke-width:2px

    %% External Access Layer
    subgraph Access
        ING_WEB["Webserver Ingress"]
        ING_FLOWER["Flower Ingress"]
    end

    %% Kubernetes Services Layer
    subgraph Services
        SVC_WEB["Webserver Service"]
        SVC_FLOWER["Flower Service"]
        SVC_REDIS["Redis Service"]
        SVC_PG["PostgreSQL Service"]
        SVC_PGB["PgBouncer Service"]
    end

    %% Airflow Components Layer
    subgraph AirflowCore
        WEBSERVER["Airflow Webserver"]:::internal
        SCHEDULER["Airflow Scheduler"]:::internal
        WORKERS["Airflow Workers"]:::internal
        TRIGGERER["Airflow Triggerer"]:::internal
        FLOWER["Flower Monitor"]:::monitoring
        PGBOUNCER["PgBouncer"]:::database
    end

    %% Data Storage Layer
    subgraph Storage
        POSTGRES["PostgreSQL"]:::database
        REDIS["Redis"]:::database
        DAGS["DAGs Storage"]:::storage
        LOGS["Logs Storage"]:::storage
    end

    %% Configuration and Sync Layer
    subgraph Config
        CONFIG["Configuration"]:::config
        SYNC["Sync Components"]:::config
        RBAC["RBAC Components"]:::config
        MONITOR["Monitoring"]:::monitoring
    end

    %% Connections
    ING_WEB --> SVC_WEB
    ING_FLOWER --> SVC_FLOWER
    SVC_WEB --> WEBSERVER
    SVC_FLOWER --> FLOWER
    SVC_REDIS --> REDIS
    SVC_PG --> POSTGRES
    SVC_PGB --> PGBOUNCER

    WEBSERVER --> PGBOUNCER
    SCHEDULER --> PGBOUNCER
    WORKERS --> PGBOUNCER
    TRIGGERER --> PGBOUNCER
    PGBOUNCER --> POSTGRES

    SCHEDULER --> REDIS
    WORKERS --> REDIS
    FLOWER --> REDIS

    WEBSERVER --> DAGS
    SCHEDULER --> DAGS
    WORKERS --> DAGS
    
    WEBSERVER --> LOGS
    SCHEDULER --> LOGS
    WORKERS --> LOGS
    TRIGGERER --> LOGS

    CONFIG --> AirflowCore
    SYNC --> POSTGRES
    RBAC --> AirflowCore
    MONITOR --> WEBSERVER

    %% Click events for component mapping
    click WEBSERVER "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/webserver/webserver-deployment.yaml"
    click SCHEDULER "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/scheduler/scheduler-deployment.yaml"
    click WORKERS "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/worker/worker-statefulset.yaml"
    click TRIGGERER "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/triggerer/triggerer-deployment.yaml"
    click FLOWER "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/flower/flower-deployment.yaml"
    click PGBOUNCER "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/pgbouncer/pgbouncer-deployment.yaml"
    click POSTGRES "https://github.com/airflow-helm/charts/tree/main/images/postgresql-bitnami/11/alpine/"
    click DAGS "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/pvc-dags.yaml"
    click LOGS "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/pvc-logs.yaml"
    click RBAC "https://github.com/airflow-helm/charts/tree/main/charts/airflow/templates/rbac/"
    click ING_WEB "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/webserver/webserver-ingress.yaml"
    click ING_FLOWER "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/flower/flower-ingress.yaml"
    click MONITOR "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/webserver/webserver-service-monitor.yaml"
    click CONFIG "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/config/secret-config-envs.yaml"
    click SYNC "https://github.com/airflow-helm/charts/blob/main/charts/airflow/templates/sync/sync-connections-deployment.yaml"
```
