How netflix/maestro works
High-Level Architecture Components
a) Core Services:
• Maestro Server (API Layer)
• Maestro Engine (Workflow Execution)
• CockroachDB Persistence Layer
• Netflix SEL (Expression Language Engine)
• Blue for API/Server components
• Green for processing/engine components
• Yellow for persistence layer
• Purple for supporting services
Component Relationships
Following interactions:
[Client Applications] → [Maestro Server (REST API)]
• ↓
[Maestro Engine] ←→ [Netflix SEL]
• ↓
[CockroachDB Persistence]
Detailed Component Breakdown
For each major component, include subcomponents:
a) Maestro Server:
• Controllers (WorkflowController, WorkflowActionController, WorkflowInstanceController)
• Configuration Management
• Request Interceptors
• Exception Handlers
b) Maestro Engine:
• Workflow Executor
• Task Runner
• Step Runtime Manager
• Concurrency Handler
• Event Processors
• Job Handlers
c) Persistence Layer:
• DAO Layer
• Migration Management
• Transaction Handling
Cross-Cutting Concerns
• Security/Authentication
• Metrics/Monitoring
• Event Publishing
• Tracing
Key Integration Points
Highlight:
• External System Integration
• Event Publishers/Subscribers
• Database Connections
• Expression Language Integration
Deployment Context
Show:
• Service Boundaries
• Network Zones
• Load Balancers (if applicable)
• Scaling Points
Technical Details
Annotations for:
• Java 21 Runtime
• Spring Boot Framework
• CockroachDB
• Docker Container Boundaries
Scale Indicators
Note:
• Handles hundreds of thousands of workflows
• Millions of jobs daily
• Multiple user types (data scientists, engineers, analysts)
Integration boundaries
Remember to include version information and any significant configuration dependencies, as this is a production-grade system requiring careful attention to deployment details.