# wepay/waltz architecture

> Architecture diagram of the GitHub repository wepay/waltz, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wepay/waltz
- Repository: https://github.com/wepay/waltz

## Overview

Waltz is a distributed, replicated write-ahead log for transactions. The main workflow is an application submitting transactions through the client API; Waltz servers coordinate replicated partitions backed by storage nodes, while clients can retrieve transaction data. Storage also exposes administrative operations, and the tools include storage recovery. The README documents the client/server/storage cluster and ZooKeeper-based test setup. Some server and client internals were not sampled, so unverified call paths are omitted; the graph emphasizes relationships visible in excerpts or documented in the README.

Main entry points: [`waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/StorageManager.java`](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/StorageManager.java) (Storage manager), [`waltz-server/src/main/java/com/wepay/waltz/store/internal/RecoveryManagerImpl.java`](https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/store/internal/RecoveryManagerImpl.java) (Replica recovery), [`waltz-storage/src/main/java/com/wepay/waltz/storage/WaltzStorage.java`](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/WaltzStorage.java) (Storage service), [`waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageClient.java`](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageClient.java) (Storage client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Client API"]
  node_client_api["Waltz client<br/>[WaltzClient.java]"]
  node_transaction["Transaction access<br/>[Transaction.java]"]
  node_rpc_client["RPC client<br/>[RpcClient.java]"]
end

subgraph group_server["Server Replication"]
  node_waltz_servers["Waltz servers<br/>[WaltzServer.java]"]
  node_server_handler["Request handling"]
  node_replica_recovery["Replica recovery"]
  node_replica_connection["Replica connection"]
  node_replica_session["Replica session"]
end

subgraph group_storage["Storage Engine"]
  node_storage_service["Storage service<br/>[WaltzStorage.java]"]
  node_storage_manager["Storage manager"]
  node_storage_partition["Storage partition<br/>[Partition.java]"]
  node_storage_client["Storage client<br/>[StorageClient.java]"]
  node_admin_client["Storage admin"]
  node_storage_disk[("Segment storage<br/>[Segment.java]")]
end

subgraph group_protocol["Shared Protocol"]
  node_records["Transaction records<br/>[Record.java]"]
  node_request_ids["Request IDs<br/>[ReqId.java]"]
  node_storage_protocol["Storage protocol"]
end

subgraph group_operations["Operations"]
  node_cluster_admin["Storage CLI<br/>[StorageCli.java]"]
  node_recovery_tool["Offline recovery"]
end

node_application(("Application"))
node_zookeeper[("ZooKeeper")]

node_application -->|"submits transactions"| node_client_api
node_client_api -.->|"sends transactions"| node_waltz_servers
node_transaction -->|"fetches data"| node_rpc_client
node_replica_recovery -->|"uses"| node_replica_connection
node_replica_recovery -->|"uses"| node_replica_session
node_storage_service -->|"configures"| node_storage_manager
node_storage_manager -->|"opens"| node_storage_partition
node_storage_manager -.->|"manages partitions"| node_storage_disk
node_storage_client -->|"transfers records"| node_records
node_storage_client -->|"encodes messages"| node_storage_protocol
node_cluster_admin -->|"assigns partitions"| node_admin_client
node_recovery_tool -->|"reads source records"| node_admin_client
node_recovery_tool -->|"writes destination"| node_replica_connection
node_cluster_admin -->|"reads cluster metadata"| node_zookeeper
node_storage_service -.->|"uses cluster metadata"| node_zookeeper

click node_client_api "https://github.com/wepay/waltz/blob/master/waltz-client/src/main/java/com/wepay/waltz/client/WaltzClient.java"
click node_transaction "https://github.com/wepay/waltz/blob/master/waltz-client/src/main/java/com/wepay/waltz/client/Transaction.java"
click node_rpc_client "https://github.com/wepay/waltz/blob/master/waltz-client/src/main/java/com/wepay/waltz/client/internal/RpcClient.java"
click node_waltz_servers "https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/server/WaltzServer.java"
click node_server_handler "https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/server/internal/WaltzServerHandler.java"
click node_replica_recovery "https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/store/internal/RecoveryManagerImpl.java"
click node_replica_connection "https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/store/internal/ReplicaConnection.java"
click node_replica_session "https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/store/internal/ReplicaSession.java"
click node_storage_service "https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/WaltzStorage.java"
click node_storage_manager "https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/StorageManager.java"
click node_storage_partition "https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/Partition.java"
click node_storage_client "https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageClient.java"
click node_admin_client "https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageAdminClient.java"
click node_storage_disk "https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/Segment.java"
click node_records "https://github.com/wepay/waltz/blob/master/waltz-common/src/main/java/com/wepay/waltz/common/message/Record.java"
click node_request_ids "https://github.com/wepay/waltz/blob/master/waltz-common/src/main/java/com/wepay/waltz/common/message/ReqId.java"
click node_storage_protocol "https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/common/message/StorageMessageCodecV0.java"
click node_cluster_admin "https://github.com/wepay/waltz/blob/master/waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageCli.java"
click node_recovery_tool "https://github.com/wepay/waltz/blob/master/waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageRecoveryRunnable.java"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_client_api,node_transaction,node_rpc_client toneBlue
class node_waltz_servers,node_server_handler,node_replica_recovery,node_replica_connection,node_replica_session,node_zookeeper toneAmber
class node_storage_service,node_storage_manager,node_storage_partition,node_storage_client,node_admin_client,node_storage_disk toneMint
class node_records,node_request_ids,node_storage_protocol toneRose
class node_cluster_admin,node_recovery_tool,node_application toneIndigo
```

## Components

### Client API

- [Waltz client](https://github.com/wepay/waltz/blob/master/waltz-client/src/main/java/com/wepay/waltz/client/WaltzClient.java): `waltz-client/src/main/java/com/wepay/waltz/client/WaltzClient.java`
- [Transaction access](https://github.com/wepay/waltz/blob/master/waltz-client/src/main/java/com/wepay/waltz/client/Transaction.java): `waltz-client/src/main/java/com/wepay/waltz/client/Transaction.java`
- [RPC client](https://github.com/wepay/waltz/blob/master/waltz-client/src/main/java/com/wepay/waltz/client/internal/RpcClient.java): `waltz-client/src/main/java/com/wepay/waltz/client/internal/RpcClient.java`

### Server Replication

- [Waltz servers](https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/server/WaltzServer.java): `waltz-server/src/main/java/com/wepay/waltz/server/WaltzServer.java`
- [Request handling](https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/server/internal/WaltzServerHandler.java): `waltz-server/src/main/java/com/wepay/waltz/server/internal/WaltzServerHandler.java`
- [Replica recovery](https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/store/internal/RecoveryManagerImpl.java): `waltz-server/src/main/java/com/wepay/waltz/store/internal/RecoveryManagerImpl.java`
- [Replica connection](https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/store/internal/ReplicaConnection.java): `waltz-server/src/main/java/com/wepay/waltz/store/internal/ReplicaConnection.java`
- [Replica session](https://github.com/wepay/waltz/blob/master/waltz-server/src/main/java/com/wepay/waltz/store/internal/ReplicaSession.java): `waltz-server/src/main/java/com/wepay/waltz/store/internal/ReplicaSession.java`

### Storage Engine

- [Storage service](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/WaltzStorage.java): `waltz-storage/src/main/java/com/wepay/waltz/storage/WaltzStorage.java`
- [Storage manager](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/StorageManager.java): `waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/StorageManager.java`
- [Storage partition](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/Partition.java): `waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/Partition.java`
- [Storage client](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageClient.java): `waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageClient.java`
- [Storage admin](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageAdminClient.java): `waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageAdminClient.java`
- [Segment storage](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/Segment.java): `waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/Segment.java`

### Shared Protocol

- [Transaction records](https://github.com/wepay/waltz/blob/master/waltz-common/src/main/java/com/wepay/waltz/common/message/Record.java): `waltz-common/src/main/java/com/wepay/waltz/common/message/Record.java`
- [Request IDs](https://github.com/wepay/waltz/blob/master/waltz-common/src/main/java/com/wepay/waltz/common/message/ReqId.java): `waltz-common/src/main/java/com/wepay/waltz/common/message/ReqId.java`
- [Storage protocol](https://github.com/wepay/waltz/blob/master/waltz-storage/src/main/java/com/wepay/waltz/storage/common/message/StorageMessageCodecV0.java): `waltz-storage/src/main/java/com/wepay/waltz/storage/common/message/StorageMessageCodecV0.java`

### Operations

- [Storage CLI](https://github.com/wepay/waltz/blob/master/waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageCli.java): `waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageCli.java`
- [Offline recovery](https://github.com/wepay/waltz/blob/master/waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageRecoveryRunnable.java): `waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageRecoveryRunnable.java`

### Other components

- **Application**
- **ZooKeeper**

## Connections

- Application → Waltz client: submits transactions
- Waltz client → Waltz servers: sends transactions
- Transaction access → RPC client: fetches data (evidence: `waltz-client/src/main/java/com/wepay/waltz/client/Transaction.java`)
- Replica recovery → Replica connection: uses (evidence: `waltz-server/src/main/java/com/wepay/waltz/store/internal/RecoveryManagerImpl.java`)
- Replica recovery → Replica session: uses (evidence: `waltz-server/src/main/java/com/wepay/waltz/store/internal/RecoveryManagerImpl.java`)
- Storage service → Storage manager: configures (evidence: `waltz-storage/src/main/java/com/wepay/waltz/storage/WaltzStorage.java`)
- Storage manager → Storage partition: opens (evidence: `waltz-storage/src/main/java/com/wepay/waltz/storage/server/internal/StorageManager.java`)
- Storage manager → Segment storage: manages partitions
- Storage client → Transaction records: transfers records (evidence: `waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageClient.java`)
- Storage client → Storage protocol: encodes messages (evidence: `waltz-storage/src/main/java/com/wepay/waltz/storage/client/StorageClient.java`)
- Storage CLI → Storage admin: assigns partitions (evidence: `waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageCli.java`)
- Offline recovery → Storage admin: reads source records (evidence: `waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageRecoveryRunnable.java`)
- Offline recovery → Replica connection: writes destination (evidence: `waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageRecoveryRunnable.java`)
- Storage CLI → ZooKeeper: reads cluster metadata (evidence: `waltz-tools/src/main/java/com/wepay/waltz/tools/storage/StorageCli.java`)
- Storage service → ZooKeeper: uses cluster metadata
