# devtool5/distributed-kv-store architecture

> Architecture diagram of the GitHub repository devtool5/distributed-kv-store, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/devtool5/distributed-kv-store
- Repository: https://github.com/devtool5/distributed-kv-store

## Overview

This is a gRPC distributed key-value store. A client sends GET/PUT/DELETE to the stateless gateway; the gateway hashes each key to a shard and retries shard nodes, while each shard’s Raft leader commits writes for replicated application to durable state. The README documents elections, recovery, WAL, persistent Raft logs, snapshots, and replicated idempotency; those modules are represented, though their detailed wiring was not sampled. The sampled code confirms gateway forwarding, shard-node request handling, duplicate lookup, and peer RPC responsibilities.

Main entry points: [`src/main/java/com/distributedkv/raft/RaftNode.java`](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/RaftNode.java) (Raft node), [`src/main/java/com/distributedkv/raft/snapshot/SnapshotManager.java`](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/snapshot/SnapshotManager.java) (Snapshot manager), [`src/main/java/com/distributedkv/sharding/ShardRouter.java`](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/sharding/ShardRouter.java) (Shard router), [`src/main/java/com/distributedkv/service/KVStoreServiceImpl.java`](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/service/KVStoreServiceImpl.java) (Node KV service)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Client API"]
  node_client_api["KV client<br/>[KVStoreClient.java]"]
end

subgraph group_gateway["Gateway routing"]
  node_gateway_server["Gateway server<br/>[GatewayServer.java]"]
  node_routing_service["Request service"]
  node_shard_router["Shard router<br/>[ShardRouter.java]"]
  node_hash_ring["Hash ring"]
  node_topology["Shard topology<br/>[ShardTopology.java]"]
  node_shard_client["Shard RPC client<br/>[ShardClient.java]"]
end

subgraph group_consensus["Shard consensus"]
  node_kv_service["Node KV service"]
  node_raft_node["Raft node<br/>[RaftNode.java]"]
  node_raft_service["Raft RPC service"]
  node_peer_manager["Peer manager"]
  node_peer_client["Peer RPC client<br/>[PeerClient.java]"]
  node_raft_log["Raft log<br/>[RaftLog.java]"]
  node_applier["State applier"]
  node_idempotency[("Duplicate table")]
end

subgraph group_durability["Durable state"]
  node_kv_store[("KV state")]
  node_wal[("Write-ahead log<br/>[WriteAheadLog.java]")]
  node_raft_store[("Raft log store")]
  node_snapshot_manager["Snapshot manager"]
  node_snapshot_store[("Snapshot store")]
end

node_user(("Application client"))

node_user -->|"issues operations"| node_client_api
node_client_api -->|"gRPC requests"| node_gateway_server
node_gateway_server -->|"serves requests"| node_routing_service
node_routing_service -->|"routes operations"| node_shard_router
node_shard_router -->|"selects shard"| node_hash_ring
node_shard_router -->|"reads shard nodes"| node_topology
node_shard_router -->|"forwards and retries"| node_shard_client
node_shard_client -->|"KV RPC"| node_kv_service
node_kv_service -->|"submits writes"| node_raft_node
node_kv_service -->|"reads GET"| node_kv_store
node_kv_service -->|"checks duplicates"| node_idempotency
node_raft_node -->|"appends commands"| node_raft_log
node_raft_node -->|"dispatches peer RPCs"| node_peer_manager
node_peer_manager -->|"manages clients"| node_peer_client
node_peer_client -->|"vote and append RPCs"| node_raft_service
node_raft_service -->|"handles Raft RPCs"| node_raft_node
node_raft_node -->|"persists log"| node_raft_store
node_raft_node -->|"applies committed entries"| node_applier
node_applier -->|"updates state"| node_kv_store
node_applier -->|"applies request records"| node_idempotency
node_kv_store -->|"persists mutations"| node_wal
node_snapshot_manager -->|"captures state"| node_kv_store
node_snapshot_manager -->|"saves snapshots"| node_snapshot_store
node_snapshot_manager -->|"compacts persisted log"| node_raft_store
node_snapshot_manager -->|"compacts in-memory log"| node_raft_log

click node_client_api "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/client/KVStoreClient.java"
click node_gateway_server "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/server/GatewayServer.java"
click node_routing_service "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/service/RequestRoutingServiceImpl.java"
click node_shard_router "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/sharding/ShardRouter.java"
click node_hash_ring "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/sharding/ConsistentHashRing.java"
click node_topology "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/sharding/ShardTopology.java"
click node_shard_client "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/cluster/ShardClient.java"
click node_kv_service "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/service/KVStoreServiceImpl.java"
click node_raft_node "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/RaftNode.java"
click node_raft_service "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/service/RaftServiceImpl.java"
click node_peer_manager "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/cluster/PeerClientManager.java"
click node_peer_client "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/cluster/PeerClient.java"
click node_raft_log "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/log/RaftLog.java"
click node_applier "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/replication/StateMachineApplier.java"
click node_idempotency "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/idempotency/DuplicateRequestTable.java"
click node_kv_store "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/storage/PersistentKVStore.java"
click node_wal "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/storage/wal/WriteAheadLog.java"
click node_raft_store "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/log/persistence/FileRaftLogStore.java"
click node_snapshot_manager "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/snapshot/SnapshotManager.java"
click node_snapshot_store "https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/snapshot/FileSnapshotStore.java"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_client_api,node_user toneBlue
class node_gateway_server,node_routing_service,node_shard_router,node_hash_ring,node_topology,node_shard_client toneAmber
class node_kv_service,node_raft_node,node_raft_service,node_peer_manager,node_peer_client,node_raft_log,node_applier,node_idempotency toneMint
class node_kv_store,node_wal,node_raft_store,node_snapshot_manager,node_snapshot_store toneRose
```

## Components

### Client API

- [KV client](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/client/KVStoreClient.java): `src/main/java/com/distributedkv/client/KVStoreClient.java`

### Gateway routing

- [Gateway server](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/server/GatewayServer.java): `src/main/java/com/distributedkv/server/GatewayServer.java`
- [Request service](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/service/RequestRoutingServiceImpl.java): `src/main/java/com/distributedkv/service/RequestRoutingServiceImpl.java`
- [Shard router](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/sharding/ShardRouter.java): `src/main/java/com/distributedkv/sharding/ShardRouter.java`
- [Hash ring](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/sharding/ConsistentHashRing.java): `src/main/java/com/distributedkv/sharding/ConsistentHashRing.java`
- [Shard topology](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/sharding/ShardTopology.java): `src/main/java/com/distributedkv/sharding/ShardTopology.java`
- [Shard RPC client](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/cluster/ShardClient.java): `src/main/java/com/distributedkv/cluster/ShardClient.java`

### Shard consensus

- [Node KV service](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/service/KVStoreServiceImpl.java): `src/main/java/com/distributedkv/service/KVStoreServiceImpl.java`
- [Raft node](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/RaftNode.java): `src/main/java/com/distributedkv/raft/RaftNode.java`
- [Raft RPC service](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/service/RaftServiceImpl.java): `src/main/java/com/distributedkv/service/RaftServiceImpl.java`
- [Peer manager](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/cluster/PeerClientManager.java): `src/main/java/com/distributedkv/cluster/PeerClientManager.java`
- [Peer RPC client](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/cluster/PeerClient.java): `src/main/java/com/distributedkv/cluster/PeerClient.java`
- [Raft log](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/log/RaftLog.java): `src/main/java/com/distributedkv/raft/log/RaftLog.java`
- [State applier](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/replication/StateMachineApplier.java): `src/main/java/com/distributedkv/raft/replication/StateMachineApplier.java`
- [Duplicate table](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/idempotency/DuplicateRequestTable.java): `src/main/java/com/distributedkv/idempotency/DuplicateRequestTable.java`

### Durable state

- [KV state](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/storage/PersistentKVStore.java): `src/main/java/com/distributedkv/storage/PersistentKVStore.java`
- [Write-ahead log](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/storage/wal/WriteAheadLog.java): `src/main/java/com/distributedkv/storage/wal/WriteAheadLog.java`
- [Raft log store](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/log/persistence/FileRaftLogStore.java): `src/main/java/com/distributedkv/raft/log/persistence/FileRaftLogStore.java`
- [Snapshot manager](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/snapshot/SnapshotManager.java): `src/main/java/com/distributedkv/raft/snapshot/SnapshotManager.java`
- [Snapshot store](https://github.com/devtool5/distributed-kv-store/blob/main/src/main/java/com/distributedkv/raft/snapshot/FileSnapshotStore.java): `src/main/java/com/distributedkv/raft/snapshot/FileSnapshotStore.java`

### Other components

- **Application client**

## Connections

- Application client → KV client: issues operations
- KV client → Gateway server: gRPC requests
- Gateway server → Request service: serves requests
- Request service → Shard router: routes operations
- Shard router → Hash ring: selects shard
- Shard router → Shard topology: reads shard nodes
- Shard router → Shard RPC client: forwards and retries
- Shard RPC client → Node KV service: KV RPC
- Node KV service → Raft node: submits writes
- Node KV service → KV state: reads GET
- Node KV service → Duplicate table: checks duplicates
- Raft node → Raft log: appends commands
- Raft node → Peer manager: dispatches peer RPCs
- Peer manager → Peer RPC client: manages clients
- Peer RPC client → Raft RPC service: vote and append RPCs
- Raft RPC service → Raft node: handles Raft RPCs
- Raft node → Raft log store: persists log
- Raft node → State applier: applies committed entries
- State applier → KV state: updates state
- State applier → Duplicate table: applies request records
- KV state → Write-ahead log: persists mutations
- Snapshot manager → KV state: captures state
- Snapshot manager → Snapshot store: saves snapshots
- Snapshot manager → Raft log store: compacts persisted log
- Snapshot manager → Raft log: compacts in-memory log