How devtool5/distributed-kv-store works
This is a gRPC distributed key-value store. A client sends GET/PUT/DELETE to the stateless gateway; the gateway hashes each key to a shard and retries shard nodes, while each shard’s Raft leader commits writes for replicated application to durable state. The README documents elections, recovery, WAL, persistent Raft logs, snapshots, and replicated idempotency; those modules are represented, though their detailed wiring was not sampled. The sampled code confirms gateway forwarding, shard-node request handling, duplicate lookup, and peer RPC responsibilities.
Main entry points: RaftNode.java Raft node · SnapshotManager.java Snapshot manager · ShardRouter.java Shard router · KVStoreServiceImpl.java Node KV service
Components 21
Client API
- KV client
src/main/java/com/distributedkv/client/KVStoreClient.java
Gateway routing
- Gateway server
src/main/java/com/distributedkv/server/GatewayServer.java - Request service
src/main/java/com/distributedkv/service/RequestRoutingServiceImpl.java - Shard router
src/main/java/com/distributedkv/sharding/ShardRouter.java - Hash ring
src/main/java/com/distributedkv/sharding/ConsistentHashRing.java - Shard topology
src/main/java/com/distributedkv/sharding/ShardTopology.java - Shard RPC client
src/main/java/com/distributedkv/cluster/ShardClient.java
Shard consensus
- Node KV service
src/main/java/com/distributedkv/service/KVStoreServiceImpl.java - Raft node
src/main/java/com/distributedkv/raft/RaftNode.java - Raft RPC service
src/main/java/com/distributedkv/service/RaftServiceImpl.java - Peer manager
src/main/java/com/distributedkv/cluster/PeerClientManager.java - Peer RPC client
src/main/java/com/distributedkv/cluster/PeerClient.java - Raft log
src/main/java/com/distributedkv/raft/log/RaftLog.java - State applier
src/main/java/com/distributedkv/raft/replication/StateMachineApplier.java - Duplicate table
src/main/java/com/distributedkv/idempotency/DuplicateRequestTable.java
Durable state
- KV state
src/main/java/com/distributedkv/storage/PersistentKVStore.java - Write-ahead log
src/main/java/com/distributedkv/storage/wal/WriteAheadLog.java - Raft log store
src/main/java/com/distributedkv/raft/log/persistence/FileRaftLogStore.java - Snapshot manager
src/main/java/com/distributedkv/raft/snapshot/SnapshotManager.java - Snapshot store
src/main/java/com/distributedkv/raft/snapshot/FileSnapshotStore.java
Other components
- Application client
Connections 25
- Application client → KV client: issues operations
- KV client → Gateway server: gRPC requests
- Gateway server → Request service: serves requests
- Request service → Shard router: routes operations
- Shard router → Hash ring: selects shard
- Shard router → Shard topology: reads shard nodes
- Shard router → Shard RPC client: forwards and retries
- Shard RPC client → Node KV service: KV RPC
- Node KV service → Raft node: submits writes
- Node KV service → KV state: reads GET
- Node KV service → Duplicate table: checks duplicates
- Raft node → Raft log: appends commands
- Raft node → Peer manager: dispatches peer RPCs
- Peer manager → Peer RPC client: manages clients
- Peer RPC client → Raft RPC service: vote and append RPCs
- Raft RPC service → Raft node: handles Raft RPCs
- Raft node → Raft log store: persists log
- Raft node → State applier: applies committed entries
- State applier → KV state: updates state
- State applier → Duplicate table: applies request records
- KV state → Write-ahead log: persists mutations
- Snapshot manager → KV state: captures state
- Snapshot manager → Snapshot store: saves snapshots
- Snapshot manager → Raft log store: compacts persisted log
- Snapshot manager → Raft log: compacts in-memory log