How azure/durabletask works
Main Components
• Core Orchestration Engine (DurableTask.Core): This is the heart of the framework. It contains definitions for orchestrations, activities, command handling, error handling, middleware, history events, and tracking. Several subdirectories (e.g., History, Logging, Middleware, Serializing) indicate its responsibilities.
• Persistence Providers: There are multiple “backend” projects that implement storage and messaging through different technologies:
– DurableTask.AzureStorage: Implements orchestration state and messaging using Azure Storage (queues, tables, blobs).
– DurableTask.ServiceBus: Uses Service Bus queues for messaging and Azure Storage for tracking state.
– DurableTask.AzureServiceFabric: Uses Azure Service Fabric’s Reliable Collections as the persistence mechanism.
– (Other providers such as DurableTask.Netherite and DurableTask.SqlServer are mentioned in the README even if not fully visible in the file tree.)
• Telemetry and Monitoring: There is a dedicated project for Application Insights integration (DurableTask.ApplicationInsights) as well as telemetry documentation and sample integrations.
• Client, Dispatcher, and Execution Components: The core library also includes components like task activity dispatchers, orchestration executors, scheduling proxies, and work item dispatchers which are essential to the task lifecycle.
Relationships and Interactions
• The Clients (or external service invocations) submit workflows to the Core Orchestration Engine.
• When an orchestration is scheduled, the engine will hand over tasks to dispatchers, which then interact with one of the available persistence providers based on configuration.
• Each provider abstracts the storage/messaging layer – for example, AzureStorage uses queues/tables/blobs, and ServiceBus uses message queues and tables – to persist orchestration state, messages, and history events.
• Telemetry components hook into orchestration and provider workflows, emitting diagnostic events or activity traces.
• There is also a testing harness and an emulator which can simulate orchestration execution in an in-memory configuration.
Architectural Patterns and Design Principles
• Pluggability/Extensibility: The persistence layer is abstracted so that developers can swap or add new providers (as signified by separate NuGet packages and folder separation).
• Layered Architecture: The project cleanly separates orchestration logic from persistence details, telemetry, and integration testing.
• Asynchronous and Resilient Execution: The framework uses async/await patterns for long-running operations and includes robust exception and error handling patterns.
• Middleware Pipeline: The use of pipeline concepts (e.g., DispatchMiddlewarePipeline) suggests that functionality is extended via middleware components.