# srv1n/dagger architecture

> Architecture diagram of the GitHub repository srv1n/dagger, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/srv1n/dagger
- Repository: https://github.com/srv1n/dagger

## Overview

Based on our analysis of the file tree and README, this project is a Rust library called “Dagger” that provides a robust engine for executing directed acyclic graphs (DAGs). Its purpose is to orchestrate task flows through either static (YAML-based) definitions or dynamic, agent-driven workflows. In addition, it supports a Task Agent System for handling more complex, persistent and agent-specific task execution. The project is structured into several main parts:

##### Core Library (src)
- Implements the core DAG functionality, including modules for dag_flow, pubsub, and taskagent.
- Contains fundamental components such as the DagExecutor, node actions, cache management, and node registry.
- Manages key elements like dependency resolution, error handling and concurrent execution (leveraging Tokio and tracing).

##### Macros Library (dagger-macros)
- Offers macros (e.g., #[action], #[task_agent], #[task_builder]) that reduce boilerplate and add functionalities like JSON schema generation.
- Provides a layer of abstraction for registering actions with more metadata and introspection capability.

##### Examples
- Several example directories demonstrate the various usage scenarios: basic agent flows, simple task execution, YAML-based DAGs, dynamic supervisor-driven flows, and a full-fledged Task Agent System.
- Some examples include configuration files (e.g., pipeline.yaml) and even a sample database state for the task manager.

##### Additional Files
- Documentation files (README.md, PubSub.md) describe usage, technical stack, and examples.
- Configuration files like Cargo.toml in different directories show how dependencies are set up for different components and examples.

1. Core Engine:
- Sub-components such as Action Registration, Dependency Management, and Cache Management.
2. Workflow Types:
- Two main execution modes: “YAML-based Static Workflows” and “Dynamic Agent-Driven Flows”. Indicate that both types use the common DAG execution infrastructure.
- For the dynamic flows, emphasize the “Supervisor” role that can add nodes at runtime, and for the static workflows, show the loading of YAML (pipeline.yaml) definitions.
3. Task Agent System:
- Specify a separate component for handling tasks with persistence, agent-specific execution, state management (using a database like the one shown in taskmanager, e.g., “example_state.db”), and task claiming/retry logic.
4. Dagger Macros:

B. Determine Relationships and Data Flow:
- That the Action Registry (potentially implemented within the DagExecutor) connects to both statically defined actions and macro-enhanced actions.
- How the Cache component interacts with each node/action, storing inputs/outputs and intermediate results.
- For the Task Agent System, indicate the flow from Task submission through state persistence, agent claiming, execution, and job completion.

C. Highlight Architectural Patterns and Technologies:
- Microservices-like separation within the library by isolating the workflow execution engine, macro functionalities, and the task agent system.
- Asynchronous execution with Tokio, error handling strategies (retries, timeouts), and usage of immutable shared data (via RwLock/Arc) in several modules.

- For a full-stack style overview, clearly separate “Frontend” (if any – in this context, the examples act as user applications) from the “Backend” logic (the core DAG engine). Emphasize the API layer where actions are registered and executed.
- Since this is an open-source library with multiple integration points, focus on the extensibility points (action registration, macros, and task agents).
- Use annotations to note that the system supports both static and dynamic workflows, providing flexibility in how workflows are defined and executed.

- Clear labels for each component (e.g., “DagExecutor”, “Action Registry”, “Cache”, “Supervisor”, “TaskAgent”, “dagger-macros”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Core Engine"
        engine["DAG Engine (DagExecutor)"]:::core
        actionRegistry["Action Registration"]:::coreSub
        dependencyMgmt["Dependency Management"]:::coreSub
        cacheMgmt["Cache Management"]:::coreSub
        pubsub["PubSub System"]:::coreSub
    end

    subgraph "Workflow Execution"
        staticWf["YAML-based Static Workflow"]:::workflow
        dynamicWf["Dynamic Agent-Driven Workflow"]:::workflow
        supervisor["Supervisor"]:::workflow
    end

    subgraph "Task Agent System"
        taskAgent["Task Agent Execution"]:::task
        persistent["Task Persistence (example_state.db)"]:::task
    end

    subgraph "Auxiliary Library"
        macros["Dagger Macros"]:::macro
    end

    subgraph "Example Applications"
        examples["Usage Examples"]:::example
    end

    %% Relationships between components
    examples -->|"triggers"| staticWf
    examples -->|"triggers"| dynamicWf
    examples -->|"uses"| taskAgent

    staticWf -->|"loads YAML"| engine
    dynamicWf -->|"initiates"| engine
    supervisor -->|"manages"| dynamicWf

    engine -->|"orchestrates"| taskAgent
    taskAgent -->|"stores data in"| persistent

    engine -->|"integrates with"| actionRegistry
    engine -->|"integrates with"| dependencyMgmt
    engine -->|"integrates with"| cacheMgmt
    engine -->|"integrates with"| pubsub

    macros -->|"enhances"| actionRegistry

    %% Click Events
    click engine "https://github.com/srv1n/dagger/blob/main/src/dag_flow/dag_flow.rs"
    click actionRegistry "https://github.com/srv1n/dagger/blob/main/src/registry.rs"
    click pubsub "https://github.com/srv1n/dagger/blob/main/src/pubsub/pubsub.rs"
    click taskAgent "https://github.com/srv1n/dagger/blob/main/src/taskagent/taskagent.rs"
    click staticWf "https://github.com/srv1n/dagger/blob/main/examples/dag_flow/pipeline.yaml"
    click supervisor "https://github.com/srv1n/dagger/blob/main/examples/deepresearch/src/agents/supervisor.rs"
    click macros "https://github.com/srv1n/dagger/blob/main/dagger-macros/src/lib.rs"
    click persistent "https://github.com/srv1n/dagger/tree/main/examples/taskmanager"

    %% Styles
    classDef core fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef coreSub fill:#B0E0E6,stroke:#000,stroke-width:1px;
    classDef workflow fill:#FFD700,stroke:#000,stroke-width:2px;
    classDef task fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef macro fill:#98FB98,stroke:#000,stroke-width:2px;
    classDef example fill:#FFDEAD,stroke:#000,stroke-width:2px;
```
