How apache/fluss works
Apache Fluss is a streaming-storage system that unifies low-latency table access with lakehouse storage. The principal workflow is an engine or application writing records to Fluss tables, then reading streaming or historical data through scans and lake integrations; administrators also manage cluster and table metadata. The repository includes client APIs, server-side coordination and tablet services, lake-format integrations, and Flink adapters. Some central implementations are not excerpted, so uncertain internal calls are omitted; README-documented capabilities are retained without asserting unsupported wiring.
Main entry points: TabletService.java Tablet service · PluginLakeStorageWrapper.java Lake storage contract · FlussTable.java Table API · FlussConnection.java Client connection
Components 20
Client access
- Client connection
fluss-client/src/main/java/org/apache/fluss/client/FlussConnection.java - Admin operations
fluss-client/src/main/java/org/apache/fluss/client/admin/FlussAdmin.java - Table API
fluss-client/src/main/java/org/apache/fluss/client/table/FlussTable.java - Table writers
fluss-client/src/main/java/org/apache/fluss/client/table/writer/AppendWriterImpl.java - Table scanners
fluss-client/src/main/java/org/apache/fluss/client/table/scanner/log/LogScannerImpl.java - Point lookup
fluss-client/src/main/java/org/apache/fluss/client/lookup/TypedLookuperImpl.java - Offset initialization
fluss-client/src/main/java/org/apache/fluss/client/initializer/OffsetsInitializer.java
Fluss services
- Coordinator
fluss-server/src/main/java/org/apache/fluss/server/coordinator/CoordinatorService.java - Tablet service
fluss-server/src/main/java/org/apache/fluss/server/tablet/TabletService.java - KV management
fluss-server/src/main/java/org/apache/fluss/server/kv/KvManager.java - Replica management
fluss-server/src/main/java/org/apache/fluss/server/replica/ReplicaManager.java - Remote log storage
fluss-server/src/main/java/org/apache/fluss/server/log/remote/DefaultRemoteLogStorage.java
Lakehouse storage
- Lake storage contract
fluss-common/src/main/java/org/apache/fluss/lake/lakestorage/PluginLakeStorageWrapper.java - Paimon integration
fluss-lake/fluss-lake-paimon/src/main/java/org/apache/fluss/lake/paimon/PaimonLakeStorage.java - Iceberg integration
fluss-lake/fluss-lake-iceberg/src/main/java/org/apache/fluss/lake/iceberg/IcebergLakeStorage.java - Lance integration
fluss-lake/fluss-lake-lance/src/main/java/org/apache/fluss/lake/lance/LanceLakeStorage.java - Hudi compaction
fluss-lake/fluss-lake-hudi/src/main/java/org/apache/fluss/lake/hudi/tiering/HudiCompactionService.java - Lake tiering
fluss-common/src/main/java/org/apache/fluss/lake/lakestorage/LakeStorage.java
Compute integrations
- Flink integration
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/source/reader/fetcher/FlinkSourceFetcherManager.java
Other components
- Engine or app
Connections 23
- Engine or app → Client connection: connects
- Client connection → Table API: opens tables
- Client connection → Admin operations: administers
- Table API → Table writers: writes through
- Table API → Table scanners: scans through
- Table API → Point lookup: looks up
- Table scanners → Offset initialization: starts at
- Table writers → Tablet service: ingests records
- Table scanners → Tablet service: reads records
- Point lookup → Tablet service: queries tables
- Admin operations → Coordinator: manages metadata
- Coordinator → Tablet service: coordinates
- Tablet service → KV management: serves state
- Tablet service → Replica management: uses replicas
- Tablet service → Remote log storage: uses remote logs
- Coordinator → Lake storage contract: configures tiering
- Lake storage contract → Paimon integration: supports
fluss-lake/fluss-lake-paimon/src/main/java/org/apache/fluss/lake/paimon/PaimonLakeStorage.java - Lake storage contract → Iceberg integration: supports
fluss-lake/fluss-lake-iceberg/src/main/java/org/apache/fluss/lake/iceberg/IcebergLakeStorage.java - Lake storage contract → Lance integration: supports
fluss-lake/fluss-lake-lance/src/main/java/org/apache/fluss/lake/lance/LanceLakeStorage.java - Lake tiering → Lake storage contract: creates tiering
fluss-common/src/main/java/org/apache/fluss/lake/lakestorage/PluginLakeStorageWrapper.java - Lake tiering → Hudi compaction: compacts data
- Flink integration → Client connection: integrates with
- Engine or app → Flink integration: processes data