# mxsm/rocketmq-rust architecture

> Architecture diagram of the GitHub repository mxsm/rocketmq-rust, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mxsm/rocketmq-rust
- Repository: https://github.com/mxsm/rocketmq-rust

## Overview

##### Main Components
- Name Server (rocketmq-namesrv): Acts as the registry and routing center to manage broker registrations and provide broker metadata to clients.
- Broker (rocketmq-broker): The central component that processes client commands, handles message dispatching, and communicates with the storage layer.
- Client (rocketmq-client): The SDK that provides interfaces for producers and consumers to send, receive, and manage messages.
- Message Store (rocketmq-store): Responsible for durable storage, including the commit log, consume queues, and indexing (e.g., index files, file-based stores).
- Remoting Module (rocketmq-remoting): Implements the network communication protocols between components (broker–client, name server–broker, etc.) with asynchronous, non-blocking design.
- Additional Modules: There are supporting utilities and libraries such as rocketmq-common (shared constants, protocol details, and utilities), rocketmq-filter, rocketmq-macros (for code generation), and auxiliary tools/CLI/TUI modules that support administrative functions.

##### Relationships and Interactions
- The Client sends messages (produce requests) and receives messages (consume requests) through the Remoting layer to the Broker.
- The Broker communicates with the Name Server to register its state, get routing configurations, and update metadata.
- The Broker interacts with the Message Store to write, flush, query, and manage messages.
- The Remoting module underpins all of these communications—abstracting the network and protocol handling between components.
- Optionally, additional components such as Proxy or Tiered Store can be represented if they later become part of the project roadmap.

##### Architectural Patterns and Design Principles
- Layered Architecture: Separate responsibilities into networking (rocketmq-remoting), business logic (rocketmq-broker, rocketmq-client), and persistence (rocketmq-store).
- Modular Design: Each module (nameserver, broker, client, store) is isolated, with a common library (rocketmq-common) used for shared data, protocols, and utilities.
- Asynchronous, Non-blocking Communication: Highlight that the system leverages Rust’s asynchronous capabilities (with tokio or similar runtimes) to achieve high concurrency.
- Extensibility: Incorporate the idea that modules like command processing, filtering, and message validation are designed for extension and customization (as seen in the specialized directories for filters, hooks, and macros).

##### Relevant Technologies and Frameworks
- Rust’s language features, particularly its memory safety and concurrency, are critical to the design.
- Cargo (Rust’s build system) is used for modularization and dependency management.
- Integration with external tools (documented via CI workflows in .github) shows the project is built with mature DevOps practices.
- The separation with distinct binaries (e.g., rocketmq-namesrv-rust, rocketmq-broker-rust) emphasizes the independent deployment of components.

- Since this is a core middleware system, give emphasis to the data flow and interactions between the message producers, brokers, and consumers.
- Clearly separate server components (Namesrv, Broker, Store) from the client applications (rocketmq-client, CLI, TUI).
- Also, include the remoting or network edge as a bridge connecting all server and client parts.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        A["Client (rocketmq-client)"]:::client
        B["CLI Tool (rocketmq-cli)"]:::client
        C["TUI Module (rocketmq-tui)"]:::client
    end

    %% Communication Layer
    subgraph "Communication Layer"
        D["Remoting Module (rocketmq-remoting)"]:::remoting
    end

    %% Broker Layer
    subgraph "Broker Layer"
        E["Broker (rocketmq-broker)"]:::server
    end

    %% Metadata & Persistence
    subgraph "Metadata & Persistence"
        F["Name Server (rocketmq-namesrv)"]:::server
        G["Message Store (rocketmq-store)"]:::persistence
    end

    %% Supporting Utilities
    subgraph "Supporting Utilities"
        H["Common Library (rocketmq-common)"]:::utility
        I["Filter Module (rocketmq-filter)"]:::utility
        J["Macros Module (rocketmq-macros)"]:::utility
        K["Administrative Tools (rocketmq-tools)"]:::utility
    end

    %% Connections between layers
    A -->|"produce/consume"| D
    B -->|"admin_commands"| D
    C -->|"admin_commands"| D

    D -->|"message_delivery"| E

    E -->|"register/update"| F
    F -->|"metadata_sync"| E

    E -->|"persist_messages"| G

    K -->|"admin_control"| E

    %% Supporting utilities usage (dotted lines indicate shared resources)
    H --- E
    H --- F
    H --- D

    I --- E
    J --- E

    %% Click Events
    click A "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-client/"
    click B "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-cli/"
    click C "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-tui/"
    click D "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-remoting/"
    click E "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-broker/"
    click F "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-namesrv/"
    click G "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-store/"
    click H "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-common/"
    click I "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-filter/"
    click J "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-macros/"
    click K "https://github.com/mxsm/rocketmq-rust/tree/main/rocketmq-tools/"

    %% Styles
    classDef client fill:#A6CEE3,stroke:#333,stroke-width:2px;
    classDef server fill:#B2DF8A,stroke:#333,stroke-width:2px;
    classDef remoting fill:#FB9A99,stroke:#333,stroke-width:2px;
    classDef persistence fill:#FDBF6F,stroke:#333,stroke-width:2px;
    classDef utility fill:#CAB2D6,stroke:#333,stroke-width:2px;
```
