# apache/flink-connector-jdbc architecture

> Architecture diagram of the GitHub repository apache/flink-connector-jdbc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apache/flink-connector-jdbc
- Repository: https://github.com/apache/flink-connector-jdbc

## Overview

##### Project type and purpose
- This is an open-source library project: the official Apache Flink JDBC Connector.
- It is structured as a multi-module Maven project that provides:
- A core module with the Flink JDBC connector API and implementation
- A suite of vendor-specific submodules (MySQL, Postgres, Oracle, etc.) that plug into the core via SPI
- Architecture and backward-compatibility test modules
- Documentation and tooling modules

##### Key modules and their roles
- flink-connector-jdbc-core
- Defines the main runtime components: JdbcSource, JdbcSink, builders, committers, enumerators, readers, dialect interfaces, catalog abstractions, connection providers, transaction managers, and utilities.
- Declares SPI interfaces (org.apache.flink.connector.jdbc.core.database.JdbcFactory) and uses META-INF/services to wire vendor factories.
- flink-connector-jdbc-architecture
- Contains ArchUnit tests enforcing architectural layering and package boundaries on production code.
- flink-connector-jdbc-backward-compatibility
- Contains tests that verify compatibility with previous connector versions across DataStream and Table/SQL APIs.
- Vendor-specific connector modules (flink-connector-jdbc-mysql, ‑postgres, ‑oracle, etc.)
- Each provides an implementation of the JdbcFactory SPI plus dialect, catalog, and type-mapping classes.
- They depend on flink-connector-jdbc-core and register themselves via META-INF/services.
- docs/ and docs/content
- Site content and connector user guides (JDBC DataStream / Table connectors).
- tools/
- CI scripts, Maven checkstyle/release configurations.

##### Architectural relationships and data flow
- At build time: core → architecture and backward-compat tests run against core (and indirectly vendor modules for compatibility).
- At runtime in Flink:
- Flink job code invokes the public API (JdbcSource/JdbcSink) defined in core.
- Core discovers and loads a vendor’s JdbcFactory via Java’s ServiceLoader and delegates dialect-specific behavior (SQL generation, type mapping, JDBC URL, catalog integration) to that factory.
- Core’s runtime components manage connection providers, split enumeration, record reading/emission, statement execution, and commit/transaction lifecycle.

- Use C4 Component or UML Component view.
- Core (blue)
- Tests (orange)
- Vendor connectors (green)
- Docs and Tools (gray)
- From tests → modules under test
- From vendor connectors → core
- From core → ServiceLoader → vendor connectors
- From Flink runtime (external) → core
- SPI interfaces and service registration (META-INF/services).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Modules"
        subgraph Core
            Core["flink-connector-jdbc-core"]:::core
            SPI["META-INF/services"]:::spi
        end
        subgraph "Vendor Connectors"
            MySQL["flink-connector-jdbc-mysql"]:::vendor
            Postgres["flink-connector-jdbc-postgres"]:::vendor
            Oracle["flink-connector-jdbc-oracle"]:::vendor
            DB2["flink-connector-jdbc-db2"]:::vendor
            CrateDB["flink-connector-jdbc-cratedb"]:::vendor
            OceanBase["flink-connector-jdbc-oceanbase"]:::vendor
            SQLServer["flink-connector-jdbc-sqlserver"]:::vendor
            Trino["flink-connector-jdbc-trino"]:::vendor
        end
    end

    subgraph "Tests"
        Arch["flink-connector-jdbc-architecture"]:::test
        Backward["flink-connector-jdbc-backward-compatibility"]:::test
    end

    subgraph "Docs & Tools"
        Docs["Documentation"]:::doc
        CI["CI Scripts"]:::tool
        MavenTool["Maven Checkstyle & Config"]:::tool
        Release["Release Tooling"]:::tool
    end

    Flink["Flink Runtime"]:::external

    Arch -->|tests| Core
    Backward -->|tests| Core
    Backward -->|tests| MySQL
    Backward -->|tests| Postgres
    Backward -->|tests| Oracle
    Backward -->|tests| DB2
    Backward -->|tests| CrateDB
    Backward -->|tests| OceanBase
    Backward -->|tests| SQLServer
    Backward -->|tests| Trino

    MySQL -->|depends on| Core
    Postgres -->|depends on| Core
    Oracle -->|depends on| Core
    DB2 -->|depends on| Core
    CrateDB -->|depends on| Core
    OceanBase -->|depends on| Core
    SQLServer -->|depends on| Core
    Trino -->|depends on| Core

    Core -->|"loads plugin"| SPI
    SPI -->|"provides factory"| MySQL
    SPI -->|"provides factory"| Postgres
    SPI -->|"provides factory"| Oracle
    SPI -->|"provides factory"| DB2
    SPI -->|"provides factory"| CrateDB
    SPI -->|"provides factory"| OceanBase
    SPI -->|"provides factory"| SQLServer
    SPI -->|"provides factory"| Trino

    Flink -->|runtime uses| Core

    click Core "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-core/"
    click Arch "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-architecture/"
    click Backward "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-backward-compatibility/"
    click MySQL "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-mysql/"
    click Postgres "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-postgres/"
    click Oracle "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-oracle/"
    click DB2 "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-db2/"
    click CrateDB "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-cratedb/"
    click OceanBase "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-oceanbase/"
    click SQLServer "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-sqlserver/"
    click Trino "https://github.com/apache/flink-connector-jdbc/tree/main/flink-connector-jdbc-trino/"
    click Docs "https://github.com/apache/flink-connector-jdbc/tree/main/docs/content/"
    click CI "https://github.com/apache/flink-connector-jdbc/tree/main/tools/ci/"
    click MavenTool "https://github.com/apache/flink-connector-jdbc/tree/main/tools/maven/"
    click Release "https://github.com/apache/flink-connector-jdbc/tree/main/tools/releasing/"

    classDef core fill:#bbdefb,stroke:#0d47a1,color:#0d47a1
    classDef vendor fill:#c8e6c9,stroke:#1b5e20,color:#1b5e20
    classDef test fill:#ffe0b2,stroke:#e65100,color:#e65100
    classDef doc fill:#eeeeee,stroke:#9e9e9e,color:#616161
    classDef tool fill:#e0e0e0,stroke:#757575,color:#424242
    classDef external fill:#f3e5f5,stroke:#6a1b9a,color:#4a148c
    classDef spi fill:#fce4ec,stroke:#880e4f,color:#880e4f
```
