# apache/iggy architecture

> Architecture diagram of the GitHub repository apache/iggy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apache/iggy
- Repository: https://github.com/apache/iggy

## Overview

Apache Iggy is a persistent, multi-transport message-streaming platform. The main workflow is a producer or consumer using an SDK to send or poll messages through a server transport, with partition storage providing persistence; management, consumer coordination, and integrations are additional product branches. The tree and README support these capabilities, but most server, shard, and connector implementation files were not sampled, so those links are shown as unverified rather than asserted calls.

Main entry points: [`core/server/src/main.rs`](https://github.com/apache/iggy/blob/master/core/server/src/main.rs) (Iggy server), [`core/partitions/src/partition_storage.rs`](https://github.com/apache/iggy/blob/master/core/partitions/src/partition_storage.rs) (Partition storage), [`core/sdk/src/tcp/tcp_client.rs`](https://github.com/apache/iggy/blob/master/core/sdk/src/tcp/tcp_client.rs) (Rust SDK), [`core/shard/src/router.rs`](https://github.com/apache/iggy/blob/master/core/shard/src/router.rs) (Shard routing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_clients["Clients and management"]
  node_sdk["Rust SDK<br/>[tcp_client.rs]"]
end

subgraph group_server["Server and transports"]
  node_server["Iggy server<br/>[main.rs]"]
  node_http["HTTP API handlers<br/>[handlers.rs]"]
  node_sessions["Session management<br/>[session_manager.rs]"]
  node_protocol["Binary protocol<br/>[dispatch.rs]"]
  node_config["Server configuration<br/>[server.rs]"]
end

subgraph group_streaming["Streaming and persistence"]
  node_shardrouter["Shard routing<br/>[router.rs]"]
  node_partition["Partition storage"]
  node_offsets["Consumer offsets<br/>[offset_storage.rs]"]
  node_journal[("Durable journal<br/>[durable_storage.rs]")]
end

subgraph group_integrations["Product integrations"]
  node_kafka["Kafka gateway<br/>[server.rs]"]
  node_connectors["Connector runtime<br/>[main.rs]"]
  node_httpsource["HTTP source<br/>[routes.rs]"]
  node_deltasink["Delta sink<br/>[storage.rs]"]
  node_mcp["MCP service<br/>[mod.rs]"]
  node_mcpconfig["MCP configuration<br/>[configs.rs]"]
end

subgraph group_operations["Administration and UI"]
  node_cli["CLI commands<br/>[main.rs]"]
  node_webui["Web permissions UI"]
end

node_producer(("Producer or consumer"))

node_producer -->|"uses SDK"| node_sdk
node_sdk -.->|"connects to"| node_server
node_cli -.->|"issues commands"| node_sdk
node_webui -.->|"calls API"| node_http
node_server -.->|"serves API"| node_http
node_server -.->|"handles protocol"| node_protocol
node_server -.->|"tracks sessions"| node_sessions
node_server -.->|"loads settings"| node_config
node_kafka -.->|"bridges messages"| node_shardrouter
node_server -.->|"routes traffic"| node_shardrouter
node_shardrouter -.->|"dispatches requests"| node_partition
node_partition -.->|"persists log"| node_journal
node_partition -.->|"recovers offsets"| node_offsets
node_connectors -.->|"hosts source"| node_httpsource
node_connectors -.->|"runs sink"| node_deltasink
node_mcp -->|"uses settings"| node_mcpconfig

click node_sdk "https://github.com/apache/iggy/blob/master/core/sdk/src/tcp/tcp_client.rs"
click node_cli "https://github.com/apache/iggy/blob/master/core/cli/src/main.rs"
click node_webui "https://github.com/apache/iggy/blob/master/web/src/lib/components/PermissionsManager.svelte"
click node_server "https://github.com/apache/iggy/blob/master/core/server/src/main.rs"
click node_http "https://github.com/apache/iggy/blob/master/core/server/src/http/handlers.rs"
click node_sessions "https://github.com/apache/iggy/blob/master/core/server/src/session_manager.rs"
click node_protocol "https://github.com/apache/iggy/blob/master/core/binary_protocol/src/dispatch.rs"
click node_kafka "https://github.com/apache/iggy/blob/master/gateways/kafka/src/server.rs"
click node_shardrouter "https://github.com/apache/iggy/blob/master/core/shard/src/router.rs"
click node_partition "https://github.com/apache/iggy/blob/master/core/partitions/src/partition_storage.rs"
click node_offsets "https://github.com/apache/iggy/blob/master/core/partitions/src/offset_storage.rs"
click node_journal "https://github.com/apache/iggy/blob/master/core/journal/src/durable_storage.rs"
click node_config "https://github.com/apache/iggy/blob/master/core/configs/src/server_config/server.rs"
click node_connectors "https://github.com/apache/iggy/blob/master/core/connectors/runtime/src/main.rs"
click node_httpsource "https://github.com/apache/iggy/blob/master/core/connectors/sources/http_source/src/routes.rs"
click node_deltasink "https://github.com/apache/iggy/blob/master/core/connectors/sinks/delta_sink/src/storage.rs"
click node_mcp "https://github.com/apache/iggy/blob/master/core/ai/mcp/src/service/mod.rs"
click node_mcpconfig "https://github.com/apache/iggy/blob/master/core/ai/mcp/src/configs.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sdk toneBlue
class node_server,node_http,node_sessions,node_protocol,node_config toneAmber
class node_shardrouter,node_partition,node_offsets,node_journal toneMint
class node_kafka,node_connectors,node_httpsource,node_deltasink,node_mcp,node_mcpconfig toneRose
class node_cli,node_webui,node_producer toneIndigo
```

## Components

### Clients and management

- [Rust SDK](https://github.com/apache/iggy/blob/master/core/sdk/src/tcp/tcp_client.rs): `core/sdk/src/tcp/tcp_client.rs`

### Server and transports

- [Iggy server](https://github.com/apache/iggy/blob/master/core/server/src/main.rs): `core/server/src/main.rs`
- [HTTP API handlers](https://github.com/apache/iggy/blob/master/core/server/src/http/handlers.rs): `core/server/src/http/handlers.rs`
- [Session management](https://github.com/apache/iggy/blob/master/core/server/src/session_manager.rs): `core/server/src/session_manager.rs`
- [Binary protocol](https://github.com/apache/iggy/blob/master/core/binary_protocol/src/dispatch.rs): `core/binary_protocol/src/dispatch.rs`
- [Server configuration](https://github.com/apache/iggy/blob/master/core/configs/src/server_config/server.rs): `core/configs/src/server_config/server.rs`

### Streaming and persistence

- [Shard routing](https://github.com/apache/iggy/blob/master/core/shard/src/router.rs): `core/shard/src/router.rs`
- [Partition storage](https://github.com/apache/iggy/blob/master/core/partitions/src/partition_storage.rs): `core/partitions/src/partition_storage.rs`
- [Consumer offsets](https://github.com/apache/iggy/blob/master/core/partitions/src/offset_storage.rs): `core/partitions/src/offset_storage.rs`
- [Durable journal](https://github.com/apache/iggy/blob/master/core/journal/src/durable_storage.rs): `core/journal/src/durable_storage.rs`

### Product integrations

- [Kafka gateway](https://github.com/apache/iggy/blob/master/gateways/kafka/src/server.rs): `gateways/kafka/src/server.rs`
- [Connector runtime](https://github.com/apache/iggy/blob/master/core/connectors/runtime/src/main.rs): `core/connectors/runtime/src/main.rs`
- [HTTP source](https://github.com/apache/iggy/blob/master/core/connectors/sources/http_source/src/routes.rs): `core/connectors/sources/http_source/src/routes.rs`
- [Delta sink](https://github.com/apache/iggy/blob/master/core/connectors/sinks/delta_sink/src/storage.rs): `core/connectors/sinks/delta_sink/src/storage.rs`
- [MCP service](https://github.com/apache/iggy/blob/master/core/ai/mcp/src/service/mod.rs): `core/ai/mcp/src/service/mod.rs`
- [MCP configuration](https://github.com/apache/iggy/blob/master/core/ai/mcp/src/configs.rs): `core/ai/mcp/src/configs.rs`

### Administration and UI

- [CLI commands](https://github.com/apache/iggy/blob/master/core/cli/src/main.rs): `core/cli/src/main.rs`
- [Web permissions UI](https://github.com/apache/iggy/blob/master/web/src/lib/components/PermissionsManager.svelte): `web/src/lib/components/PermissionsManager.svelte`

### Other components

- **Producer or consumer**

## Connections

- Producer or consumer → Rust SDK: uses SDK
- Rust SDK → Iggy server: connects to
- CLI commands → Rust SDK: issues commands
- Web permissions UI → HTTP API handlers: calls API
- Iggy server → HTTP API handlers: serves API
- Iggy server → Binary protocol: handles protocol
- Iggy server → Session management: tracks sessions
- Iggy server → Server configuration: loads settings
- Kafka gateway → Shard routing: bridges messages
- Iggy server → Shard routing: routes traffic
- Shard routing → Partition storage: dispatches requests
- Partition storage → Durable journal: persists log
- Partition storage → Consumer offsets: recovers offsets (evidence: `core/partitions/src/partition_storage.rs`)
- Connector runtime → HTTP source: hosts source
- Connector runtime → Delta sink: runs sink
- MCP service → MCP configuration: uses settings (evidence: `core/ai/mcp/src/main.rs`)
