# weibocom/rill-flow architecture

> Architecture diagram of the GitHub repository weibocom/rill-flow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/weibocom/rill-flow
- Repository: https://github.com/weibocom/rill-flow

## Overview

##### High-Level Architecture Components

a) Frontend Layer:
- Flow Graph UI (flow-graph directory)
- Management UI (rill-flow-ui directory)
- As web interfaces connected to the backend via HTTP/REST

b) Backend Core Services:
- Flow Web Service (rill-flow-web)
- Flow Service Layer (rill-flow-service)
- DAG Engine (rill-flow-dag)
- Task Template Service (rill-flow-task-template)
- Flow Trigger Service (rill-flow-trigger)

c) Storage Layer:
- Redis for runtime storage
- MySQL for persistent storage
- Clear data flow paths between services and storage

d) Execution Layer:
- Executor Framework
- Sample Executors (FastAPI, Spring Boot)
- How executors interact with the core system

##### Integration Points

- OpenTelemetry integration for observability
- External service connections (e.g., AIGC services)
- API interfaces for client integration
- Kafka integration for event-driven workflows

##### Key Workflows

Main operational flows:
- DAG Definition and Deployment
- Task Execution and Orchestration
- State Management
- Error Handling and Retry Mechanisms

##### Component Details

For each major component, include:
- Input/Output interfaces
- Key responsibilities
- Dependencies on other components
- Protocol information (HTTP, gRPC, etc.)

##### Cross-Cutting Concerns

How these aspects are handled:
- Security and Authentication
- Monitoring and Metrics
- Distributed Tracing
- High Availability features

Use the following visual elements:
- Clear labels and legends
- Version information and deployment context

- The distributed nature of the system
- High-performance characteristics
- Cloud-native deployment capabilities
- Integration points for AIGC/LLM services
- Scalability aspects

Remember to maintain a balance between detail and clarity, focusing on the architectural significance rather than implementation details.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles and Classes
    classDef frontend fill:#e1f5fe,stroke:#01579b
    classDef backend fill:#e8f5e9,stroke:#2e7d32
    classDef storage fill:#fff3e0,stroke:#ff6f00
    classDef executor fill:#f3e5f5,stroke:#7b1fa2
    classDef integration fill:#fce4ec,stroke:#c2185b
    classDef api fill:#f5f5f5,stroke:#424242

    %% Frontend Layer
    subgraph Frontend
        FG["Flow Graph UI"]
        MUI["Management UI"]
    end
    class FG,MUI frontend

    %% Backend Core Services
    subgraph CoreServices
        WS["Flow Web Service"]
        SL["Flow Service Layer"]
        DAG["DAG Engine"]
        TTS["Task Template Service"]
        TRG["Flow Trigger Service"]
    end
    class WS,SL,DAG,TTS,TRG backend

    %% Storage Layer
    subgraph Storage
        RD[(Redis)]
        SQL[(MySQL)]
        SI["Storage Implementation"]
    end
    class RD,SQL,SI storage

    %% Execution Layer
    subgraph Executors
        EF["Executor Framework"]
        FE["FastAPI Executor"]
        SE["Spring Boot Executor"]
    end
    class EF,FE,SE executor

    %% Integration Components
    subgraph Integration
        OT["OpenTelemetry"]
        KF["Kafka"]
        AIGC["AIGC Services"]
    end
    class OT,KF,AIGC integration

    %% Core Components
    subgraph Core
        CU["Common Utilities"]
        IF["Interfaces"]
        IM["Implementation"]
    end
    class CU,IF,IM backend

    %% Relationships
    FG --> WS
    MUI --> WS
    WS --> SL
    SL --> DAG
    SL --> TTS
    SL --> TRG
    DAG --> SI
    SI --> RD
    SI --> SQL
    DAG --> EF
    EF --> FE
    EF --> SE
    TRG --> KF
    DAG --> OT
    SL --> AIGC

    %% Click Events
    click FG "https://github.com/weibocom/rill-flow/tree/main/flow-graph/"
    click MUI "https://github.com/weibocom/rill-flow/tree/main/rill-flow-ui/"
    click WS "https://github.com/weibocom/rill-flow/tree/main/rill-flow-web/"
    click SL "https://github.com/weibocom/rill-flow/tree/main/rill-flow-service/"
    click DAG "https://github.com/weibocom/rill-flow/tree/main/rill-flow-dag/"
    click TTS "https://github.com/weibocom/rill-flow/tree/main/rill-flow-task-template/"
    click TRG "https://github.com/weibocom/rill-flow/tree/main/rill-flow-trigger/"
    click SI "https://github.com/weibocom/rill-flow/tree/main/rill-flow-dag/olympicene-storage/"
    click EF "https://github.com/weibocom/rill-flow/tree/main/executors/"
    click FE "https://github.com/weibocom/rill-flow/tree/main/executors/fastapi/"
    click SE "https://github.com/weibocom/rill-flow/tree/main/executors/spring-boot/"
    click CU "https://github.com/weibocom/rill-flow/tree/main/rill-flow-common/"
    click IF "https://github.com/weibocom/rill-flow/tree/main/rill-flow-interfaces/"
    click IM "https://github.com/weibocom/rill-flow/tree/main/rill-flow-impl/"
    click OT "https://github.com/weibocom/rill-flow/blob/main/docker/opentelemetry-javaagent.jar"

    %% Legend
    subgraph Legend
        L1["Frontend Components"]:::frontend
        L2["Backend Services"]:::backend
        L3["Storage Systems"]:::storage
        L4["Executors"]:::executor
        L5["Integration Points"]:::integration
    end
```
