# bidulgi69/raft-consensus architecture

> Architecture diagram of the GitHub repository bidulgi69/raft-consensus, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bidulgi69/raft-consensus
- Repository: https://github.com/bidulgi69/raft-consensus

## Overview

##### Project Type and Purpose
- This is a distributed-systems learning tool: a lightweight Java implementation of the Raft consensus algorithm. It’s delivered as a Spring-based microservice that can be spun up in multiple Docker containers and orchestrated by Docker Compose. Its goal is to let you experiment with Raft leader election, log replication, snapshotting, membership changes, and fault-tolerance scenarios.

##### File-Structure Analysis
- Top-level build/config files:
- build.gradle / settings.gradle → Gradle build of a Spring Boot + gRPC service
- Dockerfile, docker-compose.yml / docker-compose.partition.yml → container orchestration of multiple nodes
- Makefile, tests/*.sh → automated bash scripts for multi-node scenarios
- src/main/java/alg/raft → core Raft modules (ElectionManager, LogManager, SnapshotManager, etc.)
- src/main/java/alg/raft/rpc & src/main/proto/raft.proto → gRPC interface definitions and service implementation (RaftGrpcService)
- src/main/java/alg/raft/api → HTTP controllers (OperationController for client commands; TestController for test orchestration)
- src/main/java/alg/raft/configuration → Spring Boot configuration and server bootstrap
- src/main/java/alg/raft/db → LocalDatabase layer for persisting log entries, snapshots, and membership state
- src/main/resources/application.yml → runtime configuration (timeouts, ports, thresholds)

##### README Insights
- Confirms support for leader election, log replication, snapshot compaction, membership changes, and fault tolerance via iptables-driven partitions.
- Tests drive multi-node clusters to validate each Raft feature.

a. Main Components
- Test Harness (bash scripts + Docker Compose)
- Raft Node (Docker container running Spring Boot + gRPC server)
- Core Modules inside each Node:
- ElectionManager / LeaseManager (leader election)
- LogManager / SnapshotManager (log append, snapshot compaction)
- Replicator / ChannelPool (send AppendEntries / InstallSnapshot over gRPC)
- LocalDatabase (persisted state: logs, snapshots, membership)
- EventDispatcher (internal event bus)
- External Interfaces:
- gRPC API (AppendEntriesRpc, InstallSnapshotRpc)
- HTTP API (OperationController for client commands; TestController)
b. Relationships and Interactions
- Test Harness uses Docker Compose to start N containers.
- Within each container, on startup, RaftBootstrap configures beans and runs GrpcServerRunner to expose gRPC endpoints.
- Nodes communicate peer-to-peer over gRPC:
- Leader → Followers: AppendEntriesRpc, InstallSnapshotRpc
- Followers → Candidate: RequestVoteRpc (defined in proto)
- OperationController (HTTP) accepts client commands and delegates to LogManager/Repli­cator.
- SnapshotManager triggers InstallSnapshotRpc when state grows beyond threshold.
- LocalDatabase persists each log entry and snapshot.
c. Architectural Patterns / Principles
- Layered design: API layer (HTTP/gRPC) → Core consensus logic → Persistence layer
- Event-driven internal bus: components dispatch events (ReplicateEvent, LogApplyEvent, etc.)
- Dependency Injection / Inversion of Control (Spring Boot)
- Containerized microservices (one node per container)
d. Key Technologies / Frameworks
- Java + Spring Boot
- gRPC (proto definitions + generated stubs)
- Docker & Docker Compose
- Bash test scripts driving network partitions via iptables
- LocalDatabase as an embedded persistence layer (could be in-memory or file based)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Test Harness Layer
    subgraph "Test Harness" 
        TH_Scripts(["Bash Test Scripts"]):::external
        TH_Orchestration(["Docker Compose & Makefile"]):::external
    end
    click TH_Scripts "https://github.com/bidulgi69/raft-consensus/tree/main/tests"
    click TH_Orchestration "https://github.com/bidulgi69/raft-consensus/blob/main/docker-compose.yml"
    click TH_Orchestration "https://github.com/bidulgi69/raft-consensus/blob/main/docker-compose.partition.yml"
    click TH_Orchestration "https://github.com/bidulgi69/raft-consensus/tree/main/Makefile"

    %% Raft Cluster
    subgraph "Raft Cluster (N Nodes)"
        direction TB
        RN1(["RaftNode"]):::external
        RN2(["RaftNode"]):::external
        RN3(["RaftNode"]):::external
        RN_Expanded(["RaftNode (Expanded)"]):::external
    end

    %% Expanded RaftNode
    subgraph "RaftNode (Expanded)"
        direction TB
        %% API Layer
        subgraph "API Layer" 
            direction TB
            HTTP(["OperationController (HTTP)"]):::api
            TESTAPI(["TestController (HTTP)"]):::api
            GRPCSVC(["RaftGrpcService (gRPC)"]):::api
        end

        %% Core Consensus
        subgraph "Consensus Core"
            direction TB
            EMgr(["ElectionManager"]):::core
            LMgr(["LeaseManager"]):::core
            LogMgr(["LogManager"]):::core
            LogApp(["LogApplicator"]):::core
            SnapMgr(["SnapshotManager"]):::core
            Replicator(["Replicator"]):::core
            ChannelPool(["ChannelPool"]):::core
            ED(["EventDispatcher"]):::core
        end

        %% Persistence Layer
        subgraph "Persistence"
            direction TB
            DB(["LocalDatabase"]):::persistence
        end
    end

    %% Config & Bootstrap
    subgraph "Bootstrap & Config"
        RB(["RaftBootstrap"]):::core
        GSR(["GrpcServerRunner"]):::core
        RP(["RaftProperties"]):::core
        APPYML(["application.yml"]):::core
    end

    %% Event Bus
    ED -->|dispatch events| EMgr

    %% Startup Flow
    TH_Orchestration -->|start containers| RN1
    TH_Orchestration --> RN2
    TH_Orchestration --> RN3
    TH_Orchestration --> RN_Expanded
    RN_Expanded -->|bootstrap| RB --> GSR
    RB --> RP
    RB --> APPYML

    %% HTTP API interactions
    TH_Scripts -->|invoke HTTP| HTTP
    HTTP -->|client commands| LogMgr
    TESTAPI -->|test commands| EMgr

    %% Internal flows
    LogMgr -->|LogAppend| LogApp
    SnapMgr -->|snapshot trigger| Replicator
    Replicator -->|AppendEntriesRpc| GRPCSVC
    ChannelPool -->|manage channels| Replicator

    %% gRPC Peer-to-Peer
    RN_Expanded -->|AppendEntriesRpc| RN2
    RN2 -->|RequestVoteRpc| RN_Expanded
    RN_Expanded -->|InstallSnapshotRpc| RN3

    %% Persistence
    LogMgr --> DB
    SnapMgr --> DB

    %% Click events for components
    click HTTP "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/api/OperationController.java"
    click TESTAPI "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/api/TestController.java"
    click GRPCSVC "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/rpc/RaftGrpcService.java"
    click EMgr "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/ElectionManager.java"
    click LMgr "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/LeaseManager.java"
    click LogMgr "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/LogManager.java"
    click LogApp "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/LogApplicator.java"
    click SnapMgr "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/SnapshotManager.java"
    click Replicator "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/Replicator.java"
    click ChannelPool "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/ChanneledPool.java"
    click GRPCSVC "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/proto/raft.proto"
    click RB "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/configuration/RaftBootstrap.java"
    click GSR "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/configuration/GrpcServerRunner.java"
    click RP "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/configuration/RaftProperties.java"
    click APPYML "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/resources/application.yml"
    click DB "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/db/LocalDatabase.java"
    click ED "https://github.com/bidulgi69/raft-consensus/blob/main/src/main/java/alg/raft/event/EventDispatcher.java"

    %% Styles
    classDef external fill:#CCCCCC,stroke:#333,stroke-width:1px
    classDef api fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef core fill:#F9E79F,stroke:#333,stroke-width:1px
    classDef persistence fill:#ABEBC6,stroke:#333,stroke-width:1px
```
