# haraldng/omnipaxos architecture

> Architecture diagram of the GitHub repository haraldng/omnipaxos, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/haraldng/omnipaxos
- Repository: https://github.com/haraldng/omnipaxos

## Overview

##### Project Type and Purpose
- OmniPaxos is primarily a library (with supporting tools and examples) that implements a consensus algorithm similar to Raft. Its goal is to provide a simple-to-use replicated log abstraction, enabling the development of strongly consistent distributed services.
- The README emphasizes its role as a backend core component for replication, consensus, reconfiguration, and leader election with extensibility for different storage and network implementations.

##### Analyzing the File Structure
- Core Components:
- The “omnipaxos” directory contains the main consensus logic. Within it, you have modules handling ballot and leader election, consensus (omni_paxos.rs), sequence Paxos (the leader, follower modules), messaging, and utility functions.
- “omnipaxos_storage” is clearly separated as a dedicated crate for storage implementations (both in-memory and persistent), encapsulating how logs and state are safely stored.
- “omnipaxos_ui” contains a UI layer (app, render, etc.) likely used to render/log internal state or provide dashboards.
- “omnipaxos_macros” provides supportive macros intended to ease integration or reduce boilerplate across the project.
- Other Supporting Components:
- The “examples” directory (dashboard and kv_store) demonstrates how to use the library in real-world full-stack-like scenarios.
- The “docs” folder, along with GitHub workflows and configuration files, supports development, CI/CD, and documentation.
- Testing and Configuration:
- Tests are spread across the “omnipaxos/tests” directory, which reveals how functionality is validated across different scenarios.
- GitHub workflows and the Cargo.toml at multiple levels suggest a modular approach and separate compiling/packaging steps.

a. Main Components:
- Consensus Core (from omnipaxos):
- Modules: omni_paxos, sequence Paxos (leader & follower), and ballot leader election.
- Storage Module (from omnipaxos_storage):
- In-memory storage
- Persistent storage
- UI/Visualization Module (from omnipaxos_ui):
- Components such as app, render, and utility functions dedicated to rendering internal state.
- Macro/Code Generation Layer (omnipaxos_macros):
- Provides helper functionality to simplify core integration.
- Examples and Integration Points:
- Dashboard, KV store examples illustrating how the library can be incorporated into distributed systems.
- Support Layers:
- Testing, CI/CD pipelines, and documentation (not necessarily part of runtime, but critical for understanding system architecture).

b. Relationships and Interactions:
- The Consensus Core communicates with the Storage Module for log persistence and snapshotting.
- The UI module may interact with both the core and the storage layer to visually represent the current state of consensus.
- Helper macros in “omnipaxos_macros” are used by the core and possibly other crates to streamline coding practices.
- The examples demonstrate how client applications integrate with the consensus library through defined APIs, likely interacting with the core and storage components.
- There’s an implicit interaction with external systems (network, storage backends) since an OmniPaxos node is designed as a simple Rust struct which can be embedded into larger application architectures.

c. Architectural Patterns and Design Principles:
- Modularity: Each major function (consensus, storage, UI, macros) is implemented as a separate crate.
- Separation of Concerns: Clear division between core logic (consensus) and ancillary functionalities (storage, UI).
- Extensibility: The design allows users to plug in customized storage or network implementations as needed.
- Testing: Multiple dedicated test suites ensure the correct interaction among modules.

d. Relevant Technologies, Frameworks, or Libraries:
- Rust’s Cargo system is used to manage the multiple crates.
- GitHub Actions for CI/CD integration.
- The use of Rust modules and macros to define concise APIs and extend functionalities.
- Documentation integration via docs.rs and a comprehensive set of markdown docs.

##### Detailed Elements
- Each component should be clearly labeled (e.g., “OmniPaxos Core”, “Storage Module”, “UI Module”, “Macros”).
- The interactions from the Consensus Core to the Storage Module.
- How UI components retrieve and display internal state.
- How examples utilize the core library.

##### Final Advice
- Be as detailed as necessary to capture interdependencies—don’t shy away from splitting components into sub-components if it aids clarity.
- Make it clear where extensibility points exist. The design of OmniPaxos means that users can plug in alternative storage or networking components, so highlighting these interfaces would be beneficial.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Consensus Core"
        CP1["omni_paxos"]:::consensus
        CP2["sequence_paxos"]:::consensus
        CP3["Leader Election"]:::consensus
    end

    subgraph "Storage Module"
        ST1["In-Memory Storage"]:::storage
        ST2["Persistent Storage"]:::storage
    end

    subgraph "UI Module"
        UI1["App"]:::ui
        UI2["Render"]:::ui
        UI3["Utility"]:::ui
    end

    subgraph "Macro/Code Generation"
        MAC1["Macros"]:::macros
    end

    subgraph "Examples"
        EX1["Dashboard"]:::external
        EX2["KV Store"]:::external
    end

    CP1 -->|"log_persistence"| ST1
    CP1 -->|"state_update"| UI1
    MAC1 -->|"enhances"| CP1
    EX1 -->|"integrates"| CP1
    EX2 -->|"integrates"| CP1
    EX1 -->|"accesses"| ST2

    click CP1 "https://github.com/haraldng/omnipaxos/blob/master/omnipaxos/src/omni_paxos.rs"
    click CP2 "https://github.com/haraldng/omnipaxos/tree/master/omnipaxos/src/sequence_paxos/"
    click ST1 "https://github.com/haraldng/omnipaxos/blob/master/omnipaxos_storage/memory_storage.rs"
    click ST2 "https://github.com/haraldng/omnipaxos/blob/master/omnipaxos_storage/persistent_storage.rs"
    click UI1 "https://github.com/haraldng/omnipaxos/blob/master/omnipaxos_ui/src/app.rs"
    click UI2 "https://github.com/haraldng/omnipaxos/blob/master/omnipaxos_ui/src/render.rs"
    click UI3 "https://github.com/haraldng/omnipaxos/blob/master/omnipaxos_ui/src/util.rs"
    click MAC1 "https://github.com/haraldng/omnipaxos/blob/master/omnipaxos_macros/src/lib.rs"
    click EX1 "https://github.com/haraldng/omnipaxos/tree/master/examples/dashboard/"
    click EX2 "https://github.com/haraldng/omnipaxos/tree/master/examples/kv_store/"

    classDef consensus fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef storage fill:#98FB98,stroke:#333,stroke-width:2px;
    classDef ui fill:#FFA07A,stroke:#333,stroke-width:2px;
    classDef macros fill:#FFD700,stroke:#333,stroke-width:2px;
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:2px;
```
