# ryanchao2012/airfly architecture

> Architecture diagram of the GitHub repository ryanchao2012/airfly, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ryanchao2012/airfly
- Repository: https://github.com/ryanchao2012/airfly

## Overview

##### Project type and purpose
- “airfly” is an open-source Python CLI tool/library, not a full-stack app.
- It auto-generates Airflow DAG definitions (`dag.py`) from user-defined task classes.

##### File structure
- Top-level directories:
- src/airfly/cli → CLI entry and argument handling
- src/airfly/model → Task interface (AirFly class)
- src/airfly/utils, _vendor → Introspection and vendored code
- examples/ → Sample task modules demonstrating usage
- tests/ → Unit tests validating each layer
- Config & build: Makefile, pyproject.toml, GitHub workflows.

##### README for additional insights
- Describes how tasks declare operator type, params, and upstream/downstream.
- Shows the command-line options and example generated code.
- Highlights extensibility (duck-typing custom task classes).

##### Main components of the system
- CLI Layer (src/airfly/cli): parses arguments, loads modules into sys.path
- Task Discovery (model + utils): introspects Python packages, finds task classes
- Dependency Resolver (model/v1): walks `upstream`/`downstream` to build a DAG graph
- Code Generator: emits Python code with imports, DAG context, TaskGroups, operators, and `>>` links
- External Interfaces:
- User’s modules (via --modname, --path)
- Airflow operator libraries (built-in and custom)
- Output Sink: the generated `dag.py` file (stdout or redirected output)

##### Relationships and interactions
- User invokes `airfly` CLI → CLI Layer
- CLI loads target module(s) → Discovery
- Discovery yields task objects → Resolver builds graph
- Resolver hands graph → Code Generator
- Code Generator references Airflow classes → emits code → Output

##### Architectural patterns and principles
- Layered modular design for clear separation of concerns
- Duck typing for flexible task class support
- Graph traversal (DFS/BFS) to resolve dependencies
- Code templating/generation rather than runtime execution

## Diagram (Mermaid)

```mermaid
graph TD
    %% CLI Layer
    subgraph "CLI Layer"
        CLI_Main["main.py"]:::internal
        CLI_Utils["utils.py"]:::internal
    end

    %% Task Discovery Layer
    subgraph "Task Discovery"
        Task_Interface["AirFly Class (Model)"]:::internal
        Discovery_Utils["introspection utils.py"]:::internal
        Vendored["vendored helpers (_vendor)"]:::internal
    end

    %% Resolver & Code Generator
    Resolver_CodeGen["Dependency Resolver & Code Generator"]:::internal

    %% External Modules
    User_Modules["User Modules (examples)"]:::external
    Airflow["Airflow Libraries"]:::external
    Config["Config & Build"]:::external

    %% Output Artifact
    Output["Generated dag.py"]:::output

    %% Flow
    User_Modules -->|loads modules| CLI_Main
    CLI_Main -->|parses args & loads| Task_Interface
    CLI_Main -->|parses args & loads| Discovery_Utils
    CLI_Main -->|parses args & loads| Vendored
    Task_Interface --> Resolver_CodeGen
    Discovery_Utils --> Resolver_CodeGen
    Vendored --> Resolver_CodeGen
    Resolver_CodeGen -->|references| Airflow
    Resolver_CodeGen -->|emits code| Output
    Config --> CLI_Main

    %% Click Events
    click CLI_Main "https://github.com/ryanchao2012/airfly/blob/main/src/airfly/cli/main.py"
    click CLI_Utils "https://github.com/ryanchao2012/airfly/blob/main/src/airfly/cli/utils.py"
    click Task_Interface "https://github.com/ryanchao2012/airfly/blob/main/src/airfly/model/__init__.py"
    click Discovery_Utils "https://github.com/ryanchao2012/airfly/blob/main/src/airfly/utils.py"
    click Vendored "https://github.com/ryanchao2012/airfly/tree/main/src/airfly/_vendor/"
    click Resolver_CodeGen "https://github.com/ryanchao2012/airfly/blob/main/src/airfly/model/v1.py"
    click User_Modules "https://github.com/ryanchao2012/airfly/tree/main/examples/"
    click Output "https://github.com/ryanchao2012/airfly/blob/main/src/airfly/cli/main.py"
    click Config "https://github.com/ryanchao2012/airfly/tree/main/Makefile"

    %% Styles
    classDef internal fill:#9f6,stroke:#333,stroke-width:1px;
    classDef external fill:#6af,stroke:#333,stroke-width:1px;
    classDef output fill:#fa6,stroke:#333,stroke-width:1px;
```
