How mxsm/rocketmq-rust works
Main Components
• Name Server (rocketmq-namesrv): Acts as the registry and routing center to manage broker registrations and provide broker metadata to clients.
• Broker (rocketmq-broker): The central component that processes client commands, handles message dispatching, and communicates with the storage layer.
• Client (rocketmq-client): The SDK that provides interfaces for producers and consumers to send, receive, and manage messages.
• Message Store (rocketmq-store): Responsible for durable storage, including the commit log, consume queues, and indexing (e.g., index files, file-based stores).
• Remoting Module (rocketmq-remoting): Implements the network communication protocols between components (broker–client, name server–broker, etc.) with asynchronous, non-blocking design.
• Additional Modules: There are supporting utilities and libraries such as rocketmq-common (shared constants, protocol details, and utilities), rocketmq-filter, rocketmq-macros (for code generation), and auxiliary tools/CLI/TUI modules that support administrative functions.
Relationships and Interactions
• The Client sends messages (produce requests) and receives messages (consume requests) through the Remoting layer to the Broker.
• The Broker communicates with the Name Server to register its state, get routing configurations, and update metadata.
• The Broker interacts with the Message Store to write, flush, query, and manage messages.
• The Remoting module underpins all of these communications—abstracting the network and protocol handling between components.
• Optionally, additional components such as Proxy or Tiered Store can be represented if they later become part of the project roadmap.
Architectural Patterns and Design Principles
• Layered Architecture: Separate responsibilities into networking (rocketmq-remoting), business logic (rocketmq-broker, rocketmq-client), and persistence (rocketmq-store).
• Modular Design: Each module (nameserver, broker, client, store) is isolated, with a common library (rocketmq-common) used for shared data, protocols, and utilities.
• Asynchronous, Non-blocking Communication: Highlight that the system leverages Rust’s asynchronous capabilities (with tokio or similar runtimes) to achieve high concurrency.
• Extensibility: Incorporate the idea that modules like command processing, filtering, and message validation are designed for extension and customization (as seen in the specialized directories for filters, hooks, and macros).
Relevant Technologies and Frameworks
• Rust’s language features, particularly its memory safety and concurrency, are critical to the design.
• Cargo (Rust’s build system) is used for modularization and dependency management.
• Integration with external tools (documented via CI workflows in .github) shows the project is built with mature DevOps practices.
• The separation with distinct binaries (e.g., rocketmq-namesrv-rust, rocketmq-broker-rust) emphasizes the independent deployment of components.
• Since this is a core middleware system, give emphasis to the data flow and interactions between the message producers, brokers, and consumers.
• Clearly separate server components (Namesrv, Broker, Store) from the client applications (rocketmq-client, CLI, TUI).
• Also, include the remoting or network edge as a bridge connecting all server and client parts.