How codenamed22/omnidict works
Project Type & Purpose
• Type: Distributed, fault-tolerant service written in Go.
• Purpose: Expose a gRPC API for Put/Get/Join, distribute keys via consistent hashing, replicate each shard with Raft for strong consistency and leader election, and survive node failures.
File Structure
• Top-level files:
– ring.go + internal/ring/ring.go: Consistent-hash ring implementation.
– internal/store/store.go: Per-shard store logic + Raft integration.
– proto/kv.proto + generated Go files: gRPC service definition (Put/Get/Join).
– docker-compose.yml & DockerFile: Containerization & multi-node orchestration.
– design.md: High-level design notes.
• Patterns:
– Separation of concerns: API layer (ring.go), ring logic, Raft/store logic.
– Modular Go packages under /internal.
Extract Additional Insights from README
• Consistent hashing to map keys → N nodes (≤25% movement on rebalance).
• Raft group per shard (leader-based writes, log-replication).
• gRPC API surface.
• CLI test script spawns multiple nodes, kills one, verifies correctness.
Main Components & Their Relationships
a. Clients
– CLI or application code issuing gRPC calls (Put, Get, Join).
b. API Gateway / gRPC Server
– ring.go + proto handlers: receives RPCs, consults the ring.
c. Consistent-Hash Ring
– internal/ring package: determines which shard/node owns a key.
d. Node Instances (3–5 replicas)
– Each node runs:
1. A gRPC server.
2. Ring service (listens for Join, membership changes).
3. Shard(s) host:
• Raft module (etcd/HashiCorp Raft) for consensus.
• In-memory key-value store behind Raft.
e. Raft Peer Communication
– Leader election & log replication over TCP between nodes in the same shard group.
f. Cluster Membership
– Join RPC → updates ring membership → triggers rebalance.
Architectural Patterns & Design Principles
• Consistent Hashing: decouples key-to-shard mapping, minimal remapping.
• RAFT Consensus: strong consistency, leader-based writes, quorum reads/writes.
• Separation of Concerns:
– API layer (RPC)
– Ring layer (sharding + membership)
– Store layer (consensus + data)
• Containerization: Docker + docker-compose for multi-node testing.
Final Checklist
– Each component clearly labeled.
– Deployment artifacts (Docker, compose) shown externally.