# confluentinc/flink-table-api-java-examples architecture

> Architecture diagram of the GitHub repository confluentinc/flink-table-api-java-examples, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/confluentinc/flink-table-api-java-examples
- Repository: https://github.com/confluentinc/flink-table-api-java-examples

## Overview

##### Project type and purpose
- This is a Java “examples” repository demonstrating how to run Apache Flink’s Table API on Confluent Cloud. It isn’t a monolithic application or microservice suite, but rather a client-side library + example programs that delegate all execution to Confluent Cloud via REST.

##### File structure
- Top-level: Maven wrapper (`mvnw`), `pom.xml`, CI config (`service.yml`), `.github/CODEOWNERS`.
- `src/main/java/io/confluent/flink/examples/table`: a set of standalone Java `main()` classes (Example_00_HelloWorld.java through Example_08_IntegrationAndDeployment.java) that exercise different Table API features.
- `src/main/resources/cloud.properties`: default Confluent Cloud connection settings.
- `jshell-init.jsh`: JShell startup script for interactive exploration.

##### README for additional insights
- Describes how the Confluent plugin injects custom `CatalogStore`, `Planner`, `Executor` into a Flink `TableEnvironment`.
- Shows runtime flow: example code constructs a `TableEnvironment` via `ConfluentSettings`, issues Table API or SQL statements, plugin packages them into REST calls to Confluent Cloud’s Statement API, fetches results, prints or materializes them.
- Lists dependencies: Flink Table API and `confluent-flink-table-api-java-plugin`.

a. Main components to include
- Developer Environment (IDE or CLI) & Maven Build
- Example Programs (Java `main()` classes)
- Configuration Layer (`cloud.properties`, CLI args, env vars)
- ConfluentSettings & ConfluentTools utilities
- Flink Table API Core (org.apache.flink.table.api)
- Confluent Flink Table API Java Plugin
- Confluent Cloud REST API (Statement API & Statement Results API)
- Confluent Cloud Compute Pool (remote Flink cluster)
- Confluent Cloud Kafka Cluster & Schema Registry (source & sink)
- JShell Playground (optional interactive REPL)

b. Relationships & data flows
1. Developer invokes `mvn package` → JAR artifact
2. Developer runs `java -cp … Example_XX` or JShell script
3. Example Program loads `cloud.properties` (or CLI / env) → builds `ConfluentSettings`
4. Example Program calls `TableEnvironment.create(settings)` → Flink Table API wired with Confluent plugin
5. On each `.execute()`, `.executeSql()`, or ConfluentTools call:
- Plugin serializes Table API plan into SQL statements
→ issues HTTP POST to Confluent Cloud Statement API
→ Confluent Cloud Compute Pool processes the job (connects to Kafka topics, Schema Registry)
→ returns StatementResult via REST
6. Example Program fetches results, prints or materializes them locally

c. Architectural patterns & design principles
- Client–Server via REST: no local Flink cluster needed
- Plugin-based extension: interceptor pattern injects Confluent-specific Catalog, Planner, Executor
- Declarative pipeline definition (Table API / SQL) separated from execution engine
- Multi-layered configuration with clear precedence

d. Key technologies & libraries
- Build: Maven (`mvnw`, `pom.xml`)
- Language: Java 11+
- API: Apache Flink Table API for Java
- Plugin: `confluent-flink-table-api-java-plugin`
- Cloud: Confluent Cloud (Compute Pool, Kafka®, Schema Registry)
- Tools: JShell, ConfluentTools utility classes

##### Essential architectural elements to capture
- Separation of concerns: Configuration, Program, API, Plugin, Cloud
- Data flow: Java code → REST → remote execution → result retrieval
- Key extension point: Confluent plugin intercepting Table API calls

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Developer/Build Layer
    subgraph "Developer & Build" 
        direction TB
        DevEnv["Developer Environment (IDE/CLI)"]:::build
        MavenWrapper["Maven Wrapper"]:::build
        CI["CI & Code Ownership"]:::build
        Docs["Project Documentation"]:::build
    end

    DevEnv -->|invokes| MavenWrapper
    MavenWrapper -->|reads pom.xml| BuildArtifact["JAR Artifact"]:::build
    CI -->|ensures quality| BuildArtifact
    Docs -->|guides usage| DevEnv

    %% Client-Side Library & Runtime
    subgraph "Client-Side Library & Runtime"
        direction TB
        Examples["Example Programs\n(src/main/java/.../table)"]:::client
        Config["Configuration Layer\n(src/main/resources/cloud.properties)"]:::client
        JShell["JShell Playground\n(jshell-init.jsh)"]:::client
        Settings["ConfluentSettings & ConfluentTools"]:::client
        TableEnv["TableEnvironment"]:::client
        FlinkCore["Flink Table API Core"]:::client
        Plugin["Confluent Flink Table API Java Plugin"]:::client
    end

    BuildArtifact -->|runs| Examples
    Examples -->|loads| Config
    Examples -->|or via| JShell
    Config -->|builds| Settings
    JShell -->|builds| Settings
    Settings -->|creates| TableEnv
    TableEnv -->|uses| FlinkCore
    TableEnv -->|intercepts| Plugin

    %% Cloud Services
    subgraph "Confluent Cloud Services"
        direction TB
        StatementAPI["Statement API"]:::cloud
        ComputePool["Compute Pool\n(remote Flink cluster)"]:::cloud
        KafkaSR["Kafka Cluster & Schema Registry"]:::cloud
    end

    Plugin -->|POST SQL plan| StatementAPI
    StatementAPI -->|executes| ComputePool
    ComputePool -->|reads/writes| KafkaSR
    ComputePool -->|returns results via| StatementAPI
    StatementAPI -->|delivers results| TableEnv
    TableEnv -->|prints/materializes| Examples

    %% Click Events
    click mvnw "https://github.com/confluentinc/flink-table-api-java-examples/tree/master/mvnw"
    click mvnw.cmd "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/mvnw.cmd"
    click mavenJar "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/.mvn/wrapper/maven-wrapper.jar"
    click mavenProps "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/.mvn/wrapper/maven-wrapper.properties"
    click pom "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/pom.xml"
    click service "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/service.yml"
    click codeowners "https://github.com/confluentinc/flink-table-api-java-examples/tree/master/.github/CODEOWNERS"
    click README "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/README.md"
    click CHANGELOG "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/CHANGELOG.md"
    click LICENSE "https://github.com/confluentinc/flink-table-api-java-examples/tree/master/LICENSE"
    click gitignore "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/.gitignore"
    click Example_00 "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink/examples/table/Example_00_HelloWorld.java"
    click Example_01 "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink.examples.table/Example_01_CatalogsAndDatabases.java"
    click Example_02 "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink.examples.table/Example_02_UnboundedTables.java"
    click Example_03 "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink.examples.table/Example_03_TransformingTables.java"
    click Example_04 "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink.examples.table/Example_04_CreatingTables.java"
    click Example_05 "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink.examples.table/Example_05_TablePipelines.java"
    click Example_06 "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink.examples.table/Example_06_ValuesAndDataTypes.java"
    click Example_07 "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink.examples.table/Example_07_Changelogs.java"
    click Example_08 "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink.examples.table/Example_08_IntegrationAndDeployment.java"
    click Template "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/java/io/confluent/flink.examples.table/TableProgramTemplate.java"
    click Config "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/src/main/resources/cloud.properties"
    click JShell "https://github.com/confluentinc/flink-table-api-java-examples/blob/master/jshell-init.jsh"

    %% Styles
    classDef build fill:#AEC6CF,stroke:#000,stroke-width:1px,color:#000;
    classDef client fill:#C1E1C1,stroke:#000,stroke-width:1px,color:#000;
    classDef cloud fill:#F4A460,stroke:#000,stroke-width:1px,color:#000;
```
