# dogukannulu/docker-airflow architecture

> Architecture diagram of the GitHub repository dogukannulu/docker-airflow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dogukannulu/docker-airflow
- Repository: https://github.com/dogukannulu/docker-airflow

## Overview

##### Main Components
- Dockerized Airflow Container:
- This is the heart of the project. The Dockerfile builds upon the official Python 3.7-slim image and installs Apache Airflow along with any additional dependencies.
- Within this container, Airflow components such as the webserver, scheduler, and (when applicable) the worker processes run.
- Executor Variants:
- Database Service (PostgreSQL):
- Message Broker (Redis):
- For setups using the CeleryExecutor, a Redis container is used as the Celery broker. This should be depicted as a separate component that communicates with both the scheduler and worker containers.
- DAGs and Scripts:
- The “dags” directory holds sample DAG files (like tuto.py). This is where the user’s workflow definitions reside. Represent this as a volume mounted into the Airflow container.
- Configuration:
- The “config/airflow.cfg” and environment variable settings (as noted in the README) are critical. Show these as either a separate configuration layer or annotate the Airflow container indicating that it is driven by custom configurations.
- CI/CD and Build Components:

##### Relationships and Interactions
- That the Airflow webserver provides the UI and API endpoints, and highlight the connections between the scheduler and webserver components.

##### Architectural Patterns
- The system design leverages containerization and microservice-oriented patterns even though it’s packaged as a single image by default. The separation between configuration, storage (Postgres), and message brokering (Redis) is essential in making Airflow scalable (especially using the CeleryExecutor).
- Use of environment variables for configuration overrides (AIRFLOW__<SECTION>__<KEY>), which allows the system to be flexible and easily re-deployable.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Deployment & CI/CD
    ci["CI/CD Pipeline"]:::automation

    %% Airflow Deployment Subgraph
    subgraph "Airflow Deployment"
        conf["Configuration"]:::config
        dags["DAGs Volume"]:::volume
        airflow["Dockerized Airflow Container"]:::container
        ws["Airflow Webserver"]:::container
        sc["Airflow Scheduler"]:::container
        subgraph "Executor Variants"
            celery["CeleryExecutor"]:::executor
            local["LocalExecutor"]:::executor
        end
        wc["Airflow Worker(s)"]:::container
    end

    %% External Services
    pg["PostgreSQL Database"]:::db
    redis["Redis Broker"]:::message

    %% Connections
    ci -->|"AutomatedBuild"| airflow
    conf -->|"Uses_config/airflow.cfg"| airflow
    dags -->|"Mount_DAGs_Directory"| airflow

    airflow -->|"Hosts_UI/API"| ws
    airflow -->|"Manages_Scheduling"| sc
    ws -->|"SQL_Metadata_Connection"| pg
    sc -->|"SQL_Metadata_Connection"| pg

    %% CeleryExecutor interactions
    sc -->|"TaskQueue_via_Redis"| redis
    celery -->|"Configures_Broker"| redis
    celery -->|"Spawns_Workers"| wc
    redis -->|"Dispatches_Tasks"| wc
    wc -->|"Sends_TaskResults"| sc

    %% LocalExecutor Interaction (alternative)
    local -->|"Runs_locally_with_Airflow"| airflow

    %% Click Events
    click airflow "https://github.com/dogukannulu/docker-airflow/tree/master/Dockerfile"
    click airflow "https://github.com/dogukannulu/docker-airflow/blob/master/config/airflow.cfg"
    click airflow "https://github.com/dogukannulu/docker-airflow/blob/master/script/entrypoint.sh"
    click celery "https://github.com/dogukannulu/docker-airflow/blob/master/docker-compose-CeleryExecutor.yml"
    click local "https://github.com/dogukannulu/docker-airflow/blob/master/docker-compose-LocalExecutor.yml"
    click dags "https://github.com/dogukannulu/docker-airflow/blob/master/dags/tuto.py"
    click conf "https://github.com/dogukannulu/docker-airflow/blob/master/config/airflow.cfg"
    click ci "https://github.com/dogukannulu/docker-airflow/blob/master/.github/workflows/ci.yml"

    %% Styles
    classDef container fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef db fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
    classDef message fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
    classDef volume fill:#D5F5E3,stroke:#117A65,stroke-width:2px;
    classDef executor fill:#FAD7A0,stroke:#B9770E,stroke-width:2px;
    classDef automation fill:#D2B4DE,stroke:#6C3483,stroke-width:2px;
    classDef config fill:#FDEBD0,stroke:#A04000,stroke-width:2px;
```
