How bidulgi69/raft-consensus works
Project Type and Purpose
• This is a distributed-systems learning tool: a lightweight Java implementation of the Raft consensus algorithm. It’s delivered as a Spring-based microservice that can be spun up in multiple Docker containers and orchestrated by Docker Compose. Its goal is to let you experiment with Raft leader election, log replication, snapshotting, membership changes, and fault-tolerance scenarios.
File-Structure Analysis
• Top-level build/config files:
• build.gradle / settings.gradle → Gradle build of a Spring Boot + gRPC service
• Dockerfile, docker-compose.yml / docker-compose.partition.yml → container orchestration of multiple nodes
• Makefile, tests/*.sh → automated bash scripts for multi-node scenarios
• src/main/java/alg/raft → core Raft modules (ElectionManager, LogManager, SnapshotManager, etc.)
• src/main/java/alg/raft/rpc & src/main/proto/raft.proto → gRPC interface definitions and service implementation (RaftGrpcService)
• src/main/java/alg/raft/api → HTTP controllers (OperationController for client commands; TestController for test orchestration)
• src/main/java/alg/raft/configuration → Spring Boot configuration and server bootstrap
• src/main/java/alg/raft/db → LocalDatabase layer for persisting log entries, snapshots, and membership state
• src/main/resources/application.yml → runtime configuration (timeouts, ports, thresholds)
README Insights
• Confirms support for leader election, log replication, snapshot compaction, membership changes, and fault tolerance via iptables-driven partitions.
• Tests drive multi-node clusters to validate each Raft feature.
a. Main Components
• Test Harness (bash scripts + Docker Compose)
• Raft Node (Docker container running Spring Boot + gRPC server)
• Core Modules inside each Node:
• ElectionManager / LeaseManager (leader election)
• LogManager / SnapshotManager (log append, snapshot compaction)
• Replicator / ChannelPool (send AppendEntries / InstallSnapshot over gRPC)
• LocalDatabase (persisted state: logs, snapshots, membership)
• EventDispatcher (internal event bus)
• External Interfaces:
• gRPC API (AppendEntriesRpc, InstallSnapshotRpc)
• HTTP API (OperationController for client commands; TestController)
b. Relationships and Interactions
• Test Harness uses Docker Compose to start N containers.
• Within each container, on startup, RaftBootstrap configures beans and runs GrpcServerRunner to expose gRPC endpoints.
• Nodes communicate peer-to-peer over gRPC:
• Leader → Followers: AppendEntriesRpc, InstallSnapshotRpc
• Followers → Candidate: RequestVoteRpc (defined in proto)
• OperationController (HTTP) accepts client commands and delegates to LogManager/Replicator.
• SnapshotManager triggers InstallSnapshotRpc when state grows beyond threshold.
• LocalDatabase persists each log entry and snapshot.
c. Architectural Patterns / Principles
• Layered design: API layer (HTTP/gRPC) → Core consensus logic → Persistence layer
• Event-driven internal bus: components dispatch events (ReplicateEvent, LogApplyEvent, etc.)
• Dependency Injection / Inversion of Control (Spring Boot)
• Containerized microservices (one node per container)
d. Key Technologies / Frameworks
• Java + Spring Boot
• gRPC (proto definitions + generated stubs)
• Docker & Docker Compose
• Bash test scripts driving network partitions via iptables
• LocalDatabase as an embedded persistence layer (could be in-memory or file based)