How weibocom/rill-flow works
High-Level Architecture Components
a) Frontend Layer:
• Flow Graph UI (flow-graph directory)
• Management UI (rill-flow-ui directory)
• As web interfaces connected to the backend via HTTP/REST
b) Backend Core Services:
• Flow Web Service (rill-flow-web)
• Flow Service Layer (rill-flow-service)
• DAG Engine (rill-flow-dag)
• Task Template Service (rill-flow-task-template)
• Flow Trigger Service (rill-flow-trigger)
c) Storage Layer:
• Redis for runtime storage
• MySQL for persistent storage
• Clear data flow paths between services and storage
d) Execution Layer:
• Executor Framework
• Sample Executors (FastAPI, Spring Boot)
• How executors interact with the core system
Integration Points
• OpenTelemetry integration for observability
• External service connections (e.g., AIGC services)
• API interfaces for client integration
• Kafka integration for event-driven workflows
Key Workflows
Main operational flows:
• DAG Definition and Deployment
• Task Execution and Orchestration
• State Management
• Error Handling and Retry Mechanisms
Component Details
For each major component, include:
• Input/Output interfaces
• Key responsibilities
• Dependencies on other components
• Protocol information (HTTP, gRPC, etc.)
Cross-Cutting Concerns
How these aspects are handled:
• Security and Authentication
• Monitoring and Metrics
• Distributed Tracing
• High Availability features
Use the following visual elements:
• Clear labels and legends
• Version information and deployment context
• The distributed nature of the system
• High-performance characteristics
• Cloud-native deployment capabilities
• Integration points for AIGC/LLM services
• Scalability aspects
Remember to maintain a balance between detail and clarity, focusing on the architectural significance rather than implementation details.