# apache/iceberg-rust architecture

> Architecture diagram of the GitHub repository apache/iceberg-rust, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apache/iceberg-rust
- Repository: https://github.com/apache/iceberg-rust

## Overview

Apache Iceberg Rust is a Rust implementation of the Iceberg table format, with catalog, storage, scan/read, write/transaction, and encryption capabilities. A typical workflow resolves a table through a catalog, accesses its metadata and data files through storage, and reads or writes table data while managing snapshots. The tree and README establish the product boundaries; sampled excerpts confirm catalog interfaces, scan context, Arrow reading, REST authentication, encryption, and storage behavior. Wiring between unsampled subsystems is shown only where supported by those boundaries.

Main entry points: [`crates/iceberg/src/catalog/session.rs`](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/catalog/session.rs) (Catalog interface), [`crates/catalog/rest/src/catalog.rs`](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/catalog.rs) (REST catalog), [`crates/catalog/rest/src/client.rs`](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/client.rs) (REST client), [`crates/iceberg/src/spec/table_metadata.rs`](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/spec/table_metadata.rs) (Table metadata)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_catalog["Catalogs"]
  node_catalog_api["Catalog interface<br/>[session.rs]"]
  node_catalog_loader["Catalog loader<br/>[lib.rs]"]
  node_rest_catalog["REST catalog<br/>[catalog.rs]"]
  node_rest_client["REST client<br/>[client.rs]"]
  node_oauth["OAuth2 auth<br/>[oauth2.rs]"]
  node_glue["Glue catalog<br/>[catalog.rs]"]
  node_hms["Hive catalog<br/>[catalog.rs]"]
  node_s3tables["S3 Tables catalog<br/>[catalog.rs]"]
  node_sql_catalog["SQL catalog<br/>[catalog.rs]"]
end

subgraph group_table["Table Model"]
  node_metadata["Table metadata<br/>[table_metadata.rs]"]
  node_schema["Schema model<br/>[mod.rs]"]
  node_transactions["Transactions and snapshots<br/>[mod.rs]"]
  node_inspect["Table inspection<br/>[mod.rs]"]
end

subgraph group_data["Data Access"]
  node_scan["Scan planning<br/>[context.rs]"]
  node_arrow_reader["Arrow reader<br/>[pipeline.rs]"]
  node_writer["Data and delete writers<br/>[mod.rs]"]
  node_file_io["File I/O<br/>[file_io.rs]"]
  node_opendal["OpenDAL storage<br/>[lib.rs]"]
end

subgraph group_security["Security"]
  node_encryption["Encryption manager<br/>[manager.rs]"]
  node_kms["Key management interface<br/>[client.rs]"]
end

subgraph group_bindings["Bindings"]
  node_python["Python bindings<br/>[lib.rs]"]
end

node_consumer(("Rust or Python consumer"))
node_catalog_service(("Catalog service"))
node_object_store[("Object or file storage")]
node_kms_service(("External KMS"))

node_consumer -->|"selects catalog"| node_catalog_loader
node_consumer -->|"uses catalog"| node_catalog_api
node_catalog_loader -->|"loads implementation"| node_catalog_api
node_catalog_api -.->|"dispatches REST"| node_rest_catalog
node_catalog_api -.->|"dispatches Glue"| node_glue
node_catalog_api -.->|"dispatches HMS"| node_hms
node_catalog_api -.->|"dispatches S3 Tables"| node_s3tables
node_catalog_api -.->|"dispatches SQL"| node_sql_catalog
node_rest_catalog -->|"sends requests"| node_rest_client
node_rest_client -->|"authenticates requests"| node_oauth
node_rest_client -->|"HTTP requests"| node_catalog_service
node_rest_catalog -->|"returns table"| node_metadata
node_catalog_api -->|"returns table"| node_metadata
node_metadata -->|"defines fields"| node_schema
node_consumer -->|"requests scan"| node_scan
node_scan -->|"provides file tasks"| node_arrow_reader
node_arrow_reader -->|"reads data files"| node_file_io
node_consumer -->|"writes table data"| node_writer
node_writer -->|"writes data files"| node_file_io
node_file_io -.->|"uses storage adapter"| node_opendal
node_opendal -->|"reads and writes"| node_object_store
node_consumer -->|"commits changes"| node_transactions
node_transactions -->|"updates table"| node_catalog_api
node_consumer -->|"inspects table"| node_inspect
node_consumer -->|"uses encryption"| node_encryption
node_encryption -->|"wraps or unwraps keys"| node_kms
node_kms -.->|"key operations"| node_kms_service
node_python -->|"exposes API"| node_consumer

click node_catalog_api "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/catalog/session.rs"
click node_catalog_loader "https://github.com/apache/iceberg-rust/blob/main/crates/catalog/loader/src/lib.rs"
click node_rest_catalog "https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/catalog.rs"
click node_rest_client "https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/client.rs"
click node_oauth "https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/auth/oauth2.rs"
click node_glue "https://github.com/apache/iceberg-rust/blob/main/crates/catalog/glue/src/catalog.rs"
click node_hms "https://github.com/apache/iceberg-rust/blob/main/crates/catalog/hms/src/catalog.rs"
click node_s3tables "https://github.com/apache/iceberg-rust/blob/main/crates/catalog/s3tables/src/catalog.rs"
click node_sql_catalog "https://github.com/apache/iceberg-rust/blob/main/crates/catalog/sql/src/catalog.rs"
click node_metadata "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/spec/table_metadata.rs"
click node_schema "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/spec/schema/mod.rs"
click node_transactions "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/transaction/mod.rs"
click node_inspect "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/inspect/mod.rs"
click node_scan "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/scan/context.rs"
click node_arrow_reader "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/arrow/reader/pipeline.rs"
click node_writer "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/writer/mod.rs"
click node_file_io "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/io/file_io.rs"
click node_opendal "https://github.com/apache/iceberg-rust/blob/main/crates/storage/opendal/src/lib.rs"
click node_encryption "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/encryption/manager.rs"
click node_kms "https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/encryption/kms/client.rs"
click node_python "https://github.com/apache/iceberg-rust/blob/main/bindings/python/src/lib.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_catalog_api,node_catalog_loader,node_rest_catalog,node_rest_client,node_oauth,node_glue,node_hms,node_s3tables,node_sql_catalog toneBlue
class node_metadata,node_schema,node_transactions,node_inspect,node_object_store toneAmber
class node_scan,node_arrow_reader,node_writer,node_file_io,node_opendal toneMint
class node_encryption,node_kms toneRose
class node_python,node_consumer,node_catalog_service,node_kms_service toneIndigo
```

## Components

### Catalogs

- [Catalog interface](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/catalog/session.rs): `crates/iceberg/src/catalog/session.rs`
- [Catalog loader](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/loader/src/lib.rs): `crates/catalog/loader/src/lib.rs`
- [REST catalog](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/catalog.rs): `crates/catalog/rest/src/catalog.rs`
- [REST client](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/client.rs): `crates/catalog/rest/src/client.rs`
- [OAuth2 auth](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/auth/oauth2.rs): `crates/catalog/rest/src/auth/oauth2.rs`
- [Glue catalog](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/glue/src/catalog.rs): `crates/catalog/glue/src/catalog.rs`
- [Hive catalog](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/hms/src/catalog.rs): `crates/catalog/hms/src/catalog.rs`
- [S3 Tables catalog](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/s3tables/src/catalog.rs): `crates/catalog/s3tables/src/catalog.rs`
- [SQL catalog](https://github.com/apache/iceberg-rust/blob/main/crates/catalog/sql/src/catalog.rs): `crates/catalog/sql/src/catalog.rs`

### Table Model

- [Table metadata](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/spec/table_metadata.rs): `crates/iceberg/src/spec/table_metadata.rs`
- [Schema model](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/spec/schema/mod.rs): `crates/iceberg/src/spec/schema/mod.rs`
- [Transactions and snapshots](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/transaction/mod.rs): `crates/iceberg/src/transaction/mod.rs`
- [Table inspection](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/inspect/mod.rs): `crates/iceberg/src/inspect/mod.rs`

### Data Access

- [Scan planning](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/scan/context.rs): `crates/iceberg/src/scan/context.rs`
- [Arrow reader](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/arrow/reader/pipeline.rs): `crates/iceberg/src/arrow/reader/pipeline.rs`
- [Data and delete writers](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/writer/mod.rs): `crates/iceberg/src/writer/mod.rs`
- [File I/O](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/io/file_io.rs): `crates/iceberg/src/io/file_io.rs`
- [OpenDAL storage](https://github.com/apache/iceberg-rust/blob/main/crates/storage/opendal/src/lib.rs): `crates/storage/opendal/src/lib.rs`

### Security

- [Encryption manager](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/encryption/manager.rs): `crates/iceberg/src/encryption/manager.rs`
- [Key management interface](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/encryption/kms/client.rs): `crates/iceberg/src/encryption/kms/client.rs`

### Bindings

- [Python bindings](https://github.com/apache/iceberg-rust/blob/main/bindings/python/src/lib.rs): `bindings/python/src/lib.rs`

### Other components

- **Rust or Python consumer**
- **Catalog service**
- **Object or file storage**
- **External KMS**

## Connections

- Rust or Python consumer → Catalog loader: selects catalog
- Rust or Python consumer → Catalog interface: uses catalog
- Catalog loader → Catalog interface: loads implementation
- Catalog interface → REST catalog: dispatches REST
- Catalog interface → Glue catalog: dispatches Glue
- Catalog interface → Hive catalog: dispatches HMS
- Catalog interface → S3 Tables catalog: dispatches S3 Tables
- Catalog interface → SQL catalog: dispatches SQL
- REST catalog → REST client: sends requests
- REST client → OAuth2 auth: authenticates requests
- REST client → Catalog service: HTTP requests
- REST catalog → Table metadata: returns table
- Catalog interface → Table metadata: returns table
- Table metadata → Schema model: defines fields
- Rust or Python consumer → Scan planning: requests scan
- Scan planning → Arrow reader: provides file tasks
- Arrow reader → File I/O: reads data files
- Rust or Python consumer → Data and delete writers: writes table data
- Data and delete writers → File I/O: writes data files
- File I/O → OpenDAL storage: uses storage adapter
- OpenDAL storage → Object or file storage: reads and writes
- Rust or Python consumer → Transactions and snapshots: commits changes
- Transactions and snapshots → Catalog interface: updates table
- Rust or Python consumer → Table inspection: inspects table
- Rust or Python consumer → Encryption manager: uses encryption
- Encryption manager → Key management interface: wraps or unwraps keys
- Key management interface → External KMS: key operations
- Python bindings → Rust or Python consumer: exposes API
