How stvp/dht works
Main Components
• DHT Node: The core component (implemented in node.go) that represents a cluster member. Each node maintains its own state, participates in key distribution, and runs an HTTP server for health checks.
• Consul Agent: An external service on 127.0.0.1:8500 used by each node for blocking API queries that refresh the local state. It acts as the backbone for membership verification.
• Rendezvous Hashing Mechanism: A logical component embedded within the DHT node’s functionality that determines which node should hold a specific key.
• Background Goroutine: A process inside each node that periodically issues blocking queries to Consul for refreshing state, logging any errors encountered.
Understand the Relationships & Interactions
• Each DHT node is responsible for joining and leaving the distributed hash table. When a node joins, it registers itself by starting an HTTP server from an ephemeral port, allowing Consul to check its liveness.
• Nodes communicate indirectly by updating their state based on Consul’s responses (using blocking queries) and by using the rendezvous hashing algorithm to decide key ownership.
• The HTTP server in each node serves the health check purpose, enabling Consul to verify node availability.
Architectural Patterns & Design Principles
• Distributed System with Service Discovery: Nodes work together through the Consul API, ensuring robust and dynamic membership.
• Rendezvous (Highest Random Weight) Hashing: This minimizes key reassignment disruption when nodes join or leave the system.
• Concurrency: Use of a background goroutine in each node ensures that the node’s local hash state is periodically refreshed without blocking the primary operations.
• Error logging: Consistent logging practices are implemented to trace errors in state refresh operations.
Relevant Technologies & Frameworks
• Go: The implementation language, with its capabilities for concurrency (goroutines) being a pivotal factor.
• Consul: For distributed configuration, service discovery, and maintaining the DHT membership.
• HTTP Server: Built into each node for health checks.
• Standard Go log package: For error logging.