# airflow-plugins/airflow-dag-creation-manager-plugin architecture

> Architecture diagram of the GitHub repository airflow-plugins/airflow-dag-creation-manager-plugin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/airflow-plugins/airflow-dag-creation-manager-plugin
- Repository: https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin

## Overview

##### Project Overview and Purpose
- This project is an Apache Airflow plugin called “DAG Creation Manager Plugin.” Its purpose is to allow users to create and manage DAGs (Directed Acyclic Graphs) through a dedicated web UI.
- It extends Airflow by adding new web routes (accessible through Airflow’s Admin tab), additional configuration options (set via airflow.cfg), and custom authentication/backends as needed.

##### Main Components
a. Airflow Ecosystem (External Component)
– Airflow Web Server: The plugin integrates with Airflow’s web interface.
– Airflow Scheduler and Executor: While not a direct part of the plugin, understanding the plugin’s placement in the overall Airflow architecture is key.
b. DAG Creation Manager Plugin (Internal Plugin Component)
– Plugin Core:
- Contains main plugin logic (e.g., plugins/dcmp/dag_creation_manager_plugin.py).
- Holds core functionality for DAG creation, management, and conversion (e.g., dcmp/dag_converter.py).
– Authentication Module:
- Found under plugins/dcmp/auth.
- Contains authentication backends (e.g., password_auth.py) and views/models to support login functionalities, which are optionally used if Airflow’s authentication is enabled.
– UI Components:
- Templates: HTML files under plugins/dcmp/templates (e.g., approve.html, compare.html, edit.html, etc.) used to render the plugin’s web pages.
- Static Assets: CSS and JavaScript files under plugins/dcmp/static. Notice various JS files (like ace.js, edit.js, etc.) and CSS files used for the web UI.
– Utility and Tools:
- Utility Modules: Such as plugins/dcmp/utils.py providing common helper functions.
- Admin/Upgrade Scripts: Under the tools folder (e.g., upgradedb.py, update_superuser.py) which are used to manage database upgrades and other tasks.
– Configuration Module:
- plugins/dcmp/settings.py holds settings specific to the plugin.
- README instructions detail additional configuration required within airflow.cfg for customized behavior.

##### Relationships and Data Flow
- The Airflow Web Server loads the plugin from the configured plugins folder. This plugin registers additional web endpoints.
- User interactions with the web UI (rendered using the templates and static assets) trigger the plugin’s backend logic to create or manage DAG definitions.
- The authentication module (if enabled) intercepts and validates user credentials.
- Utility scripts and command-line tools are used for maintenance tasks, such as database upgrades.
- The configuration settings provided in airflow.cfg guide how the plugin behaves, especially around UI options, auth, and DAG line interpolation.

##### Architectural Patterns and Design Principles
- Plugin Architecture: The design leverages Airflow’s plugin system. It is modular and integrates tightly with an external system.
- MVC-like Separation: Although not a full MVC framework, you have clear separation between:
– Models (plugins/dcmp/models.py, auth/models.py)
– Views/Templates (HTML files in plugins/dcmp/templates)
– Controller Logic (views in plugins/dcmp/auth/views.py and plugin core code)
- Extensibility: The structure supports custom authentication backends and configurable DAG properties through settings.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Airflow Ecosystem"
        AW("Airflow Web Server"):::external
        ASE("Airflow Scheduler/Executor"):::external
    end

    U("User"):::external

    subgraph "DAG Creation Manager Plugin"
        subgraph "Plugin Core Modules"
            CPL("Core Plugin Logic"):::plugin
            DMC("DAG Management/Conversion"):::plugin
            PM("Plugin Models"):::plugin
        end

        subgraph "Authentication Module"
            AB("Authentication Backends"):::auth
            AM("Authentication Models"):::auth
            AV("Authentication Views"):::auth
        end

        subgraph "UI Components"
            T("Templates (Web UI)"):::ui
            SA("Static Assets (CSS/JS)"):::ui
        end

        subgraph "Utility and Tools"
            UM("Utility Modules"):::utility
            AUS("Admin/Upgrade Scripts"):::utility
        end

        CS("Plugin Specific Settings"):::config
    end

    %% External interactions
    U -->|"request"| T
    AW -->|"loads-plugin"| CPL
    AW -->|"serves"| T

    %% UI interactions with Plugin Core
    T -->|"triggers"| CPL
    SA -->|"assets-used"| CPL

    %% Authentication interactions with Plugin Core
    AB -->|"auth-check"| CPL
    AM -->|"provides-data"| CPL
    AV -->|"renders-views"| CPL

    %% Configuration dependency on Plugin Core
    CS -->|"configures"| CPL

    %% Utility and Tools integration
    UM -->|"helper-functions"| CPL
    AUS -->|"maintenance-tasks"| CPL

    %% Internal Plugin Core relationships
    CPL -->|"uses"| DMC
    CPL -->|"stores"| PM

    %% Click Events
    click CPL "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/blob/master/plugins/dcmp/dag_creation_manager_plugin.py"
    click DMC "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/blob/master/plugins/dcmp/dag_converter.py"
    click PM "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/blob/master/plugins/dcmp/models.py"
    click AB "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/blob/master/plugins/dcmp/auth/backends/password_auth.py"
    click AM "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/blob/master/plugins/dcmp/auth/models.py"
    click AV "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/blob/master/plugins/dcmp/auth/views.py"
    click T "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/tree/master/plugins/dcmp/templates/dcmp/"
    click SA "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/tree/master/plugins/dcmp/static/dcmp/"
    click UM "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/blob/master/plugins/dcmp/utils.py"
    click AUS "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/tree/master/plugins/dcmp/tools/"
    click CS "https://github.com/airflow-plugins/airflow-dag-creation-manager-plugin/blob/master/plugins/dcmp/settings.py"

    %% Styling
    classDef external fill:#cce5ff,stroke:#333,stroke-width:2px;
    classDef plugin fill:#d4edda,stroke:#333,stroke-width:2px;
    classDef auth fill:#fff3cd,stroke:#333,stroke-width:2px;
    classDef ui fill:#f8d7da,stroke:#333,stroke-width:2px;
    classDef utility fill:#d1ecf1,stroke:#333,stroke-width:2px;
    classDef config fill:#e2e3e5,stroke:#333,stroke-width:2px;

    class AW,ASE,U external;
    class CPL,DMC,PM plugin;
    class AB,AM,AV auth;
    class T,SA ui;
    class UM,AUS utility;
    class CS config;
```
