# azure/durabletask architecture

> Architecture diagram of the GitHub repository azure/durabletask, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/azure/durabletask
- Repository: https://github.com/azure/durabletask

## Overview

##### Main Components
- Core Orchestration Engine (DurableTask.Core): This is the heart of the framework. It contains definitions for orchestrations, activities, command handling, error handling, middleware, history events, and tracking. Several subdirectories (e.g., History, Logging, Middleware, Serializing) indicate its responsibilities.
- Persistence Providers: There are multiple “backend” projects that implement storage and messaging through different technologies:
– DurableTask.AzureStorage: Implements orchestration state and messaging using Azure Storage (queues, tables, blobs).
– DurableTask.ServiceBus: Uses Service Bus queues for messaging and Azure Storage for tracking state.
– DurableTask.AzureServiceFabric: Uses Azure Service Fabric’s Reliable Collections as the persistence mechanism.
– (Other providers such as DurableTask.Netherite and DurableTask.SqlServer are mentioned in the README even if not fully visible in the file tree.)
- Telemetry and Monitoring: There is a dedicated project for Application Insights integration (DurableTask.ApplicationInsights) as well as telemetry documentation and sample integrations.
- Client, Dispatcher, and Execution Components: The core library also includes components like task activity dispatchers, orchestration executors, scheduling proxies, and work item dispatchers which are essential to the task lifecycle.

##### Relationships and Interactions
- The Clients (or external service invocations) submit workflows to the Core Orchestration Engine.
- When an orchestration is scheduled, the engine will hand over tasks to dispatchers, which then interact with one of the available persistence providers based on configuration.
- Each provider abstracts the storage/messaging layer – for example, AzureStorage uses queues/tables/blobs, and ServiceBus uses message queues and tables – to persist orchestration state, messages, and history events.
- Telemetry components hook into orchestration and provider workflows, emitting diagnostic events or activity traces.
- There is also a testing harness and an emulator which can simulate orchestration execution in an in-memory configuration.

##### Architectural Patterns and Design Principles
- Pluggability/Extensibility: The persistence layer is abstracted so that developers can swap or add new providers (as signified by separate NuGet packages and folder separation).
- Layered Architecture: The project cleanly separates orchestration logic from persistence details, telemetry, and integration testing.
- Asynchronous and Resilient Execution: The framework uses async/await patterns for long-running operations and includes robust exception and error handling patterns.
- Middleware Pipeline: The use of pipeline concepts (e.g., DispatchMiddlewarePipeline) suggests that functionality is extended via middleware components.

## Diagram (Mermaid)

```mermaid
graph TD
    %% External Interfaces
    subgraph "External Interfaces"
        Client["Client/API"]:::external
    end

    %% Core Orchestration Engine
    subgraph "Core Orchestration Engine"
        CoreEngine["DurableTask.Core"]:::core
        Middleware["Middleware & Dispatchers"]:::core
    end

    %% Persistence Providers
    subgraph "Persistence Providers"
        AzureStorage["Azure Storage Provider"]:::provider
        ServiceBus["Service Bus Provider"]:::provider
        AzureServiceFabric["Azure Service Fabric Provider"]:::provider
    end

    %% Telemetry & Monitoring
    subgraph "Telemetry & Monitoring"
        AppInsights["Application Insights Integration"]:::telemetry
    end

    %% Testing & Samples
    subgraph "Testing & Samples"
        Emulator["Emulator for Testing Orchestrations"]:::external
        Samples["Samples & Usage Examples"]:::external
        Testing["Testing & Validation Harness"]:::external
    end

    %% Flows
    Client -->|"SubmitWorkflow"| CoreEngine
    CoreEngine -->|"DispatchActivities"| Middleware

    CoreEngine -->|"StoreState"| AzureStorage
    CoreEngine -->|"QueueMessages"| ServiceBus
    CoreEngine -->|"StoreState"| AzureServiceFabric

    AzureStorage -->|"CallbackRetrieveState"| CoreEngine
    ServiceBus -->|"CallbackRetrieveState"| CoreEngine
    AzureServiceFabric -->|"CallbackRetrieveState"| CoreEngine

    CoreEngine -->|"EmitDiagnostics"| AppInsights
    AzureStorage -->|"EmitDiagnostics"| AppInsights
    ServiceBus -->|"EmitDiagnostics"| AppInsights
    AzureServiceFabric -->|"EmitDiagnostics"| AppInsights

    Emulator -->|"SimulateOrchestration"| CoreEngine
    Samples -->|"DemonstrateUsage"| CoreEngine
    Testing -->|"ValidateBehaviors"| CoreEngine

    %% Click Events
    click CoreEngine "https://github.com/azure/durabletask/blob/main/src/DurableTask.Core"
    click AzureStorage "https://github.com/azure/durabletask/blob/main/src/DurableTask.AzureStorage"
    click ServiceBus "https://github.com/azure/durabletask/blob/main/src/DurableTask.ServiceBus"
    click AzureServiceFabric "https://github.com/azure/durabletask/blob/main/src/DurableTask.AzureServiceFabric"
    click AppInsights "https://github.com/azure/durabletask/blob/main/src/DurableTask.ApplicationInsights"
    click Emulator "https://github.com/azure/durabletask/blob/main/src/DurableTask.Emulator"
    click Samples "https://github.com/azure/durabletask/tree/main/samples"
    click Testing "https://github.com/azure/durabletask/blob/main/Test/DurableTask.AzureStorage.Tests"

    %% Styles
    classDef core fill:#f9c,stroke:#333,stroke-width:2px;
    classDef provider fill:#ccf,stroke:#333,stroke-width:2px;
    classDef external fill:#cfc,stroke:#333,stroke-width:2px;
    classDef telemetry fill:#ff9,stroke:#333,stroke-width:2px;
```
