How nicktindall/java-raft works
Project type and purpose
• This is a Java library (“java-raft”) implementing the Raft distributed consensus algorithm.
• It is not a user-facing front-end or full-stack app, but a core algorithm/toolkit that consumers embed in their own services.
File structure
• Single codebase under src/main/java/au/id/tindall/distalg/raft, split into sub-packages:
• rpc (client, server, clustermembership, snapshots)
• comms (Cluster, ClientConnection)
• elections
• log (entries, storage)
• replication
• serverstates (Follower, Candidate, Leader, plus membership/change/transfer)
• state (persistent state, snapshots)
• statemachine (command execution)
• processors (InboxProcessor, ReplicationProcessor, ProcessorManager, etc.)
• serialisation
• util, threading, timing
• client (sessions, pending responses) and clusterclient (ClusterClient, AdminClient)
• Tests mirror the same structure under src/test.
Key insights from the README
• Core features: leader election, log replication, snapshotting, cluster membership, linearizable semantics.
• Runtime dependency: only log4j2.
• Build/deploy via Gradle; CI via CircleCI and Jenkinsfile.
Main components
a. Client APIs
• ClientSession, PendingResponseRegistry, ClusterClient, ClusterAdminClient
b. Raft Server Node
• Server (ServerImpl) & ServerFactory
• RPC layer (AppendEntries, RequestVote, client RPCs)
• Communication layer (ClusterFactory → Cluster → ClientConnection)
• Processor framework (ProcessorManager, InboxProcessor, ReplicationProcessor, ElectionScheduler)
• Elections module (ElectionScheduler, ElectionTimeoutGenerator)
• Replication module (LogReplicator, HeartbeatScheduler, SnapshotReplicator)
• Log & Storage (Log, LogStorage, PersistentLogStorage, InMemoryLogStorage)
• Persistent State & Snapshots (PersistentState, Snapshotter)
• State machine (StateMachine, CommandExecutor)
• ServerStates (Follower, Candidate, Leader + membership/transfer handlers)
• Serialisation (ByteBufferIO, IDSerializer)
c. External services & resources
• Network (TCP/UDP or simulated via comms)
• Disk I/O (for persistent state & snapshots)
• Logging (log4j2)
Relationships & dataflows
• Client → RPC layer → Server
• Server RPC → Comms → other Server RPCs
• RPC messages → Serialisation in/out
• RPC events → InboxProcessor → ProcessorManager dispatch
• ElectionScheduler & ReplicationScheduler drive timed tasks
• Processors append to Log → Commit → apply to StateMachine
• Log entries & state → persisted to disk (LogStorage, PersistentState)
• Snapshotter triggers compaction, writes snapshots to disk
• ClusterAdminClient → RPC → membership change handlers inside ServerStates
Architectural patterns & principles
• Modular layering: API → RPC → Comms → Processing → Core Raft modules → Storage
• Factory pattern for pluggable components (ServerFactory, LogFactory, SnapshotterFactory)
• Event-driven processors (handlers invoked on message/events)
• Separation of concerns: each sub-package owns one slice of Raft logic
• In-memory vs persistent implementations for easy testing/flexibility
Essential elements to capture
• Client/ClusterClient → RPC → Server
• RPC → Comms → Network → Other RPC
• Incoming RPC → InboxProcessor → ProcessorManager
• Processors trigger ElectionScheduler, ReplicationScheduler
• Processors append entries → LogStorage → Disk
• Commit events → StateMachine.applyCommand
• Snapshotter triggered by log compaction logic
• ServerState transitions (Follower ↔ Candidate ↔ Leader)
• Factory interfaces for plugging in custom storage, threading, state machine
Final advice
• Clearly label interfaces and their implementations to highlight extensibility points.
• Runtime topology (clients, nodes, storage) rather than build scripts or test code.