# ververica/flink-sql-gateway architecture

> Architecture diagram of the GitHub repository ververica/flink-sql-gateway, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ververica/flink-sql-gateway
- Repository: https://github.com/ververica/flink-sql-gateway

## Overview

##### Project Type & Purpose
- This is a specialized middleware application that provides a REST API interface for users (or JDBC clients via the Flink JDBC driver) to interact with an Apache Flink cluster. It’s designed to submit SQL queries, manage sessions, deploy jobs, and retrieve results from Flink.
- Its main function is to serve as a gateway that converts SQL commands from external clients into operations that get executed on a Flink cluster, managing configuration and session state along the way.

##### Core Directories & Components (Based on the File Tree)
- bin: Contains startup scripts (e.g., sql-gateway.sh) that launch the gateway service.
- conf: Holds configuration files (including YAML defaults and logging properties) that define the server, session, and execution environments.
- src/main/java/com/ververica/flink/table/gateway:
- Configuration: Packages like config and config/entries (e.g., ConfigUtil.java, Environment.java) handle parsing and processing of the YAML environment/configuration.
- REST API: Sub-packages under rest and rest/handler implement the HTTP endpoints (e.g., SqlGatewayEndpoint.java), REST request handling (e.g., SessionCreateHandler, StatementExecuteHandler) and message objects.
- Operations: The operation package consists of classes that translate SQL (such as SELECT, INSERT, etc.) into executable operations. The OperationFactory and various Operation subclasses (e.g., CreateViewOperation, DDLOperation) capture this functionality.
- Context & Session: The context package (e.g., SessionContext, ExecutionContext) along with the rest/session classes provide the session lifecycle management.
- Deployment: The deployment package contains classes that manage integration with different Flink executor environments (e.g., ClusterDescriptorAdapter, ProgramDeployer) for various execution targets (standalone, Yarn, Kubernetes).
- Results and Sinks: Packages such as result and sink define the structures for returning results and submitting table/job outputs.

##### Key Relationships & Data Flows
- The REST API layer receives external requests (e.g., via SqlGatewayEndpoint), then directs them to the appropriate REST handlers.
- REST handlers use session management components (SessionManager, SessionContext) to track user sessions and manage session properties.
- Incoming SQL commands are parsed by the Operation classes (e.g., SqlCommandParser and various Operation subclasses) and then executed using the configured execution/deployment strategy.
- The configuration layer loads environment settings from YAML files. These settings drive behavior in the session, catalog, table, and deployment components.
- Deployment modules then interface with the Flink cluster (using various adapter classes) to submit and monitor jobs.

##### Architectural Patterns and Design Principles
- Modular Design: Each core responsibility (configuration, rest, operations, deployment, session management) is separated into its own package.
- Layered Architecture: The design clearly separates the API layer from the business logic (operations) and the deployment/integration layer.
- Extensibility: With easily configurable environment (YAML) and clear extension points (user-defined functions, connectors, customizable deployment strategies), the system is designed to be expanded.
- Clear Separation of Concerns: Different types of operations (DDL, DML) are handled by dedicated classes, and session management is decoupled from command execution.

a. Main Components:
- REST API Endpoint (e.g., SqlGatewayEndpoint) and its various request handlers (SessionCreateHandler, StatementExecuteHandler, etc.).
- Session Management Component (SessionManager, SessionContext, SessionID) that keeps track of user sessions.
- SQL Command Parser & Operation Factory: The layer that parses SQL commands and creates operation objects (e.g., CreateViewOperation, SelectOperation).
- Configuration Loader: A component that loads YAML environment files (via ConfigUtil and Environment classes).
- Deployment & Execution Manager: Classes (like ClusterDescriptorAdapter, ProgramDeployer) that interface with the Flink cluster.
- Result Handling: Classes that format and send back results (e.g., Result, BatchResult) and table sinks.
b. Relationships & Dependencies:
- Data flow from incoming REST requests → REST Handlers → Operation/Session Management → Deployment on Flink cluster → result/feedback.
- External dependencies: the underlying Flink cluster and optionally external JARs loaded at startup.
c. Other Visual Elements:

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Nodes
    E1("External Client"):::external
    E2("Flink Cluster"):::external

    %% Initialization Layer
    subgraph "Initialization"
        F1("Startup Scripts"):::init
    end

    %% API Layer
    subgraph "API Layer"
        A1("SqlGatewayEndpoint"):::api
        B1("REST Handlers"):::api
    end

    %% Business Logic Layer
    subgraph "Business Logic Layer"
        C1("Session Manager"):::business
        C2("SessionContext"):::business
        C3("SqlCommandParser"):::business
        C4("OperationFactory"):::business
        C5("Operation Subclasses"):::business
        C6("ConfigUtil"):::business
        C7("Environment"):::business
        C8("YAML Config"):::business
    end

    %% Integration/Deployment Layer
    subgraph "Integration/Deployment Layer"
        D1("Deployment Manager"):::deploy
        D2("Result Handler"):::deploy
        D3("Sink Components"):::deploy
    end

    %% Relationships
    F1 --> A1
    E1 --> A1
    A1 --> B1
    B1 --> C1
    B1 --> C3
    C8 --> C6
    C6 --> C1
    C6 --> C3
    C3 --> C4
    C4 --> C5
    C1 --> C2
    C4 --> D1
    D1 --> E2
    D1 --> D2
    D2 --> D3
    D2 --> E1

    %% Click Events
    click A1 "https://github.com/ververica/flink-sql-gateway/blob/master/src/main/java/com/ververica/flink/table/gateway/rest/SqlGatewayEndpoint.java"
    click B1 "https://github.com/ververica/flink-sql-gateway/tree/master/src/main/java/com/ververica/flink/table/gateway/rest/handler"
    click C1 "https://github.com/ververica/flink-sql-gateway/tree/master/src/main/java/com/ververica/flink/table/gateway/rest/session"
    click C2 "https://github.com/ververica/flink-sql-gateway/blob/master/src/main/java/com/ververica/flink/table/gateway/context/SessionContext.java"
    click C3 "https://github.com/ververica/flink-sql-gateway/blob/master/src/main/java/com/ververica/flink/table/gateway/operation/SqlCommandParser.java"
    click C4 "https://github.com/ververica/flink-sql-gateway/blob/master/src/main/java/com/ververica/flink/table/gateway/operation/OperationFactory.java"
    click C5 "https://github.com/ververica/flink-sql-gateway/tree/master/src/main/java/com/ververica/flink/table/gateway/operation"
    click C6 "https://github.com/ververica/flink-sql-gateway/blob/master/src/main/java/com/ververica/flink/table/gateway/config/ConfigUtil.java"
    click C7 "https://github.com/ververica/flink-sql-gateway/blob/master/src/main/java/com/ververica/flink/table/gateway/config/Environment.java"
    click C8 "https://github.com/ververica/flink-sql-gateway/blob/master/conf/sql-gateway-defaults.yaml"
    click D1 "https://github.com/ververica/flink-sql-gateway/tree/master/src/main/java/com/ververica/flink/table/gateway/deployment"
    click D2 "https://github.com/ververica/flink-sql-gateway/tree/master/src/main/java/com/ververica/flink/table/gateway/result"
    click D3 "https://github.com/ververica/flink-sql-gateway/tree/master/src/main/java/com/ververica/flink/table/gateway/sink"
    click F1 "https://github.com/ververica/flink-sql-gateway/blob/master/bin/sql-gateway.sh"

    %% Styles
    classDef api fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef business fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef deploy fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef external fill:#F0E68C,stroke:#000,stroke-width:2px;
    classDef init fill:#FFFFE0,stroke:#000,stroke-width:2px;
```
