How confluentinc/flink-table-api-java-examples works
Project type and purpose
• This is a Java “examples” repository demonstrating how to run Apache Flink’s Table API on Confluent Cloud. It isn’t a monolithic application or microservice suite, but rather a client-side library + example programs that delegate all execution to Confluent Cloud via REST.
File structure
• Top-level: Maven wrapper (mvnw), pom.xml, CI config (service.yml), .github/CODEOWNERS.
• src/main/java/io/confluent/flink/examples/table: a set of standalone Java main() classes (Example_00_HelloWorld.java through Example_08_IntegrationAndDeployment.java) that exercise different Table API features.
• src/main/resources/cloud.properties: default Confluent Cloud connection settings.
• jshell-init.jsh: JShell startup script for interactive exploration.
README for additional insights
• Describes how the Confluent plugin injects custom CatalogStore, Planner, Executor into a Flink TableEnvironment.
• Shows runtime flow: example code constructs a TableEnvironment via ConfluentSettings, issues Table API or SQL statements, plugin packages them into REST calls to Confluent Cloud’s Statement API, fetches results, prints or materializes them.
• Lists dependencies: Flink Table API and confluent-flink-table-api-java-plugin.
a. Main components to include
• Developer Environment (IDE or CLI) & Maven Build
• Example Programs (Java main() classes)
• Configuration Layer (cloud.properties, CLI args, env vars)
• ConfluentSettings & ConfluentTools utilities
• Flink Table API Core (org.apache.flink.table.api)
• Confluent Flink Table API Java Plugin
• Confluent Cloud REST API (Statement API & Statement Results API)
• Confluent Cloud Compute Pool (remote Flink cluster)
• Confluent Cloud Kafka Cluster & Schema Registry (source & sink)
• JShell Playground (optional interactive REPL)
b. Relationships & data flows
1. Developer invokes mvn package → JAR artifact
2. Developer runs java -cp … Example_XX or JShell script
3. Example Program loads cloud.properties (or CLI / env) → builds ConfluentSettings
4. Example Program calls TableEnvironment.create(settings) → Flink Table API wired with Confluent plugin
5. On each .execute(), .executeSql(), or ConfluentTools call:
• Plugin serializes Table API plan into SQL statements
→ issues HTTP POST to Confluent Cloud Statement API
→ Confluent Cloud Compute Pool processes the job (connects to Kafka topics, Schema Registry)
→ returns StatementResult via REST
6. Example Program fetches results, prints or materializes them locally
c. Architectural patterns & design principles
• Client–Server via REST: no local Flink cluster needed
• Plugin-based extension: interceptor pattern injects Confluent-specific Catalog, Planner, Executor
• Declarative pipeline definition (Table API / SQL) separated from execution engine
• Multi-layered configuration with clear precedence
d. Key technologies & libraries
• Build: Maven (mvnw, pom.xml)
• Language: Java 11+
• API: Apache Flink Table API for Java
• Plugin: confluent-flink-table-api-java-plugin
• Cloud: Confluent Cloud (Compute Pool, Kafka®, Schema Registry)
• Tools: JShell, ConfluentTools utility classes
Essential architectural elements to capture
• Separation of concerns: Configuration, Program, API, Plugin, Cloud
• Data flow: Java code → REST → remote execution → result retrieval
• Key extension point: Confluent plugin intercepting Table API calls