# drmagice/gmqtt architecture

> Architecture diagram of the GitHub repository drmagice/gmqtt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/drmagice/gmqtt
- Repository: https://github.com/drmagice/gmqtt

## Overview

1. **Overall shape**
   - This repository is an MQTT broker written in Go, centered around a runtime broker process plus a set of pluggable subsystems.
   - The main architectural boundary is between the **core broker** (`server/`) and the **extensions/plugins** (`plugin/`), with shared protocol/state packages underneath.

2. **Process entrypoints**
   - `cmd/gmqttd` is the broker daemon. It boots the server, loads configuration, starts network listeners, and wires in plugins.
   - `cmd/gmqctl` is a separate operator CLI, mainly for administrative/plugin-related workflows such as plugin generation.
   - `cmd/gmqttd/default_config.yml` and related config files define the broker’s runtime composition.

3. **Core broker runtime**
   - `server/` is the orchestrator: it owns client lifecycle, publish/subscribe flow, hook dispatch, rate limiting, metrics/stats, plugin registration, and API registration.
   - This package is the central integration point where incoming packets are turned into broker actions and outgoing deliveries are scheduled.

4. **Client and packet flow**
   - `pkg/packets/` contains MQTT protocol types and codecs for CONNECT, PUBLISH, SUBSCRIBE, ACKs, disconnects, auth, and properties.
   - Network traffic enters as protocol packets, is parsed into these types, then passed through server logic and hooks before being routed or acknowledged.
   - `message.go`, `session.go`, and `subscription.go` define shared domain concepts used across the broker.

5. **Subscription and publish routing**
   - Subscription matching is handled in the persistence/subscription layer, with an in-memory trie implementation and a Redis-backed option.
   - Publish processing fans out to matching subscribers, applies topic alias logic (`server/topic_alias.go`, `topicalias/`), and then enqueues deliveries according to QoS semantics.
   - Retained messages are managed separately under `retained/`, with trie-based storage for retained-topic lookup.

6. **QoS state and delivery durability**
   - The `persistence/queue/` and `persistence/unack/` packages track in-flight QoS state, delivery queues, and acknowledgements.
   - These subsystems are what let the broker resume or retry message delivery correctly, especially for QoS 1/2 flows.
   - `persistence/encoding/` provides the storage encoding layer used by Redis-backed persistence.

7. **Session persistence**
   - `persistence/session/` abstracts session storage, with both memory and Redis implementations.
   - The broker can run with ephemeral sessions in-memory or with Redis for restart-resilient session state.
   - `persistence/` is therefore a major infrastructure boundary: the broker core depends on the abstraction, not a concrete store.

8. **Plugin system**
   - `plugin/` is a first-class extension boundary. The core broker exposes hooks so plugins can observe or alter connection, auth, subscription, publish, delivery, and shutdown events.
   - Hooks are defined and invoked from the server layer; plugins can also register APIs and interact with broker state through exported server interfaces.

9. **Built-in plugins**
   - **Auth plugin**: provides username/password authentication and account management.
   - **Admin plugin**: exposes operational APIs over gRPC and REST, with generated protobuf, grpc-gateway, and Swagger artifacts.
   - **Prometheus plugin**: exports metrics.
   - **Federation plugin**: experimental clustering/membership and inter-node coordination, with protobuf/gRPC surfaces and peer logic.
   - These plugins are not just observers; some can actively publish messages, manage subscriptions, and mutate broker state.

10. **External integrations**
   - **Redis** is the primary external datastore used for persistence backends and some plugin storage.
   - **HTTP/gRPC** are used by the admin and federation surfaces.
   - **Prometheus** is the metrics sink exposed by the metrics plugin.
   - The broker also supports **TLS/SSL** and **WebSocket** endpoints, which affects the network entry layer in `cmd/gmqttd` and `server/`.

11. **Configuration and platform concerns**
   - `config/` models broker, MQTT, persistence, API, and topic-alias configuration, with tests and defaults.
   - Platform-specific code in `cmd/gmqttd/config_*.go` and `pkg/pidfile/` handles OS differences and process management details.
   - `server/limiter.go` and hooks support operational controls such as connection throttling and ACL-like behavior.

12. **Testing and generated artifacts**
   - The repo is heavily test-oriented, with package-level tests and shared test suites for persistence/session/subscription behavior.
   - A number of `.pb.go`, `grpc`, `gateway`, and `swagger` files are generated artifacts that define the broker’s plugin-facing APIs rather than core runtime logic.
   - `plugin_generate.go` and the plugin scaffolding files indicate that extension modules are meant to be generated and composed consistently.

Main entry points: [`server/server.go`](https://github.com/drmagice/gmqtt/blob/master/server/server.go) (Server), [`cmd/gmqttd/main.go`](https://github.com/drmagice/gmqtt/blob/master/cmd/gmqttd/main.go) (gmqttd), [`config/config.go`](https://github.com/drmagice/gmqtt/blob/master/config/config.go) (Config), [`server/hook.go`](https://github.com/drmagice/gmqtt/blob/master/server/hook.go) (Hooks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_runtime["Broker runtime"]
  node_cmd_gmqttd["gmqttd<br/>broker daemon<br/>[main.go]"]
  node_cmd_gmqctl["gmqctl<br/>operator CLI<br/>[main.go]"]
  node_server_core["Server<br/>broker core<br/>[server.go]"]
  node_server_client["Clients<br/>connection/session flow<br/>[client.go]"]
  node_server_publish["Publish path<br/>routing pipeline<br/>[publish_service.go]"]
  node_server_hooks["Hooks<br/>extension points<br/>[hook.go]"]
  node_server_api["API registrar<br/>api wiring<br/>[api_registrar.go]"]
end

subgraph group_group_persistence["Persistence"]
  node_persistence_session[("Sessions<br/>session store<br/>[session.go]")]
  node_persistence_subs[("Subscriptions<br/>subscription index<br/>[subscription.go]")]
  node_persistence_queue[("Queues<br/>QoS queue<br/>[queue.go]")]
  node_persistence_unack[("Unacked<br/>ack tracker<br/>[unack.go]")]
  node_retained_store[("Retained<br/>[interface.go]")]
end

subgraph group_group_plugins["Plugins"]
  node_plugin_auth["Auth plugin<br/>auth extension<br/>[auth.go]"]
  node_plugin_admin["Admin plugin<br/>management api<br/>[admin.go]"]
  node_plugin_prom["Prometheus<br/>metrics plugin<br/>[prometheus.go]"]
  node_plugin_federation["Federation<br/>cluster plugin<br/>[federation.go]"]
end

subgraph group_group_shared["Shared packages"]
  node_config_pkg["Config<br/>[config.go]"]
  node_packets_pkg["Packets<br/>mqtt codec<br/>[packets.go]"]
  node_domain_models["Domain models<br/>shared state<br/>[message.go]"]
  node_topicalias_pkg{{"Topic alias<br/>aliasing<br/>[fifo.go]"}}
end

node_cmd_gmqttd -->|"loads"| node_config_pkg
node_cmd_gmqttd -->|"starts"| node_server_core
node_cmd_gmqttd -->|"enables"| node_plugin_auth
node_cmd_gmqttd -->|"enables"| node_plugin_admin
node_cmd_gmqttd -->|"enables"| node_plugin_prom
node_cmd_gmqttd -->|"enables"| node_plugin_federation
node_cmd_gmqctl -->|"operates"| node_plugin_admin
node_cmd_gmqctl -->|"operates"| node_plugin_auth
node_server_core -->|"owns"| node_server_client
node_server_core -->|"routes"| node_server_publish
node_server_core -->|"dispatches"| node_server_hooks
node_server_core -->|"registers"| node_server_api
node_server_core -->|"uses"| node_persistence_session
node_server_core -->|"uses"| node_persistence_subs
node_server_core -->|"uses"| node_persistence_queue
node_server_core -->|"uses"| node_persistence_unack
node_server_core -->|"uses"| node_retained_store
node_server_publish -->|"parses"| node_packets_pkg
node_server_publish -->|"applies"| node_topicalias_pkg
node_server_client -->|"parses"| node_packets_pkg
node_server_client -->|"tracks"| node_domain_models
node_persistence_session -->|"configured by"| node_config_pkg
node_persistence_subs -->|"configured by"| node_config_pkg
node_persistence_queue -->|"configured by"| node_config_pkg
node_persistence_unack -->|"configured by"| node_config_pkg
node_plugin_auth -->|"hooks"| node_server_hooks
node_plugin_admin -->|"registers"| node_server_api
node_plugin_prom -->|"hooks"| node_server_hooks
node_plugin_federation -->|"hooks"| node_server_hooks
node_plugin_federation -->|"registers"| node_server_api
node_retained_store -->|"stores"| node_packets_pkg
node_server_core -->|"reads"| node_config_pkg

click node_cmd_gmqttd "https://github.com/drmagice/gmqtt/blob/master/cmd/gmqttd/main.go"
click node_cmd_gmqctl "https://github.com/drmagice/gmqtt/blob/master/cmd/gmqctl/main.go"
click node_server_core "https://github.com/drmagice/gmqtt/blob/master/server/server.go"
click node_server_client "https://github.com/drmagice/gmqtt/blob/master/server/client.go"
click node_server_publish "https://github.com/drmagice/gmqtt/blob/master/server/publish_service.go"
click node_server_hooks "https://github.com/drmagice/gmqtt/blob/master/server/hook.go"
click node_server_api "https://github.com/drmagice/gmqtt/blob/master/server/api_registrar.go"
click node_config_pkg "https://github.com/drmagice/gmqtt/blob/master/config/config.go"
click node_packets_pkg "https://github.com/drmagice/gmqtt/blob/master/pkg/packets/packets.go"
click node_domain_models "https://github.com/drmagice/gmqtt/blob/master/message.go"
click node_persistence_session "https://github.com/drmagice/gmqtt/blob/master/persistence/session/session.go"
click node_persistence_subs "https://github.com/drmagice/gmqtt/blob/master/persistence/subscription/subscription.go"
click node_persistence_queue "https://github.com/drmagice/gmqtt/blob/master/persistence/queue/queue.go"
click node_persistence_unack "https://github.com/drmagice/gmqtt/blob/master/persistence/unack/unack.go"
click node_retained_store "https://github.com/drmagice/gmqtt/blob/master/retained/interface.go"
click node_plugin_auth "https://github.com/drmagice/gmqtt/blob/master/plugin/auth/auth.go"
click node_plugin_admin "https://github.com/drmagice/gmqtt/blob/master/plugin/admin/admin.go"
click node_plugin_prom "https://github.com/drmagice/gmqtt/blob/master/plugin/prometheus/prometheus.go"
click node_plugin_federation "https://github.com/drmagice/gmqtt/blob/master/plugin/federation/federation.go"
click node_topicalias_pkg "https://github.com/drmagice/gmqtt/blob/master/topicalias/fifo/fifo.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cmd_gmqttd,node_cmd_gmqctl,node_server_core,node_server_client,node_server_publish,node_server_hooks,node_server_api toneBlue
class node_persistence_session,node_persistence_subs,node_persistence_queue,node_persistence_unack,node_retained_store toneAmber
class node_plugin_auth,node_plugin_admin,node_plugin_prom,node_plugin_federation toneMint
class node_config_pkg,node_packets_pkg,node_domain_models,node_topicalias_pkg toneRose
```

## Components

### Broker runtime

Process entrypoints and the core server that orchestrates MQTT traffic and plugins.

- [gmqttd](https://github.com/drmagice/gmqtt/blob/master/cmd/gmqttd/main.go): broker daemon · Main broker process that loads config, starts listeners, and wires plugins. · `cmd/gmqttd/main.go`
- [gmqctl](https://github.com/drmagice/gmqtt/blob/master/cmd/gmqctl/main.go): operator CLI · Administrative command-line tool, including plugin generation workflows. · `cmd/gmqctl/main.go`
- [Server](https://github.com/drmagice/gmqtt/blob/master/server/server.go): broker core · Central orchestrator for client lifecycle, hook dispatch, routing, metrics, and plugin registration. · `server/server.go`
- [Clients](https://github.com/drmagice/gmqtt/blob/master/server/client.go): connection/session flow · Client lifecycle, connection handling, and packet-facing broker state. · `server/client.go`
- [Publish path](https://github.com/drmagice/gmqtt/blob/master/server/publish_service.go): routing pipeline · Publish processing, QoS handling, topic aliasing, and delivery scheduling. · `server/publish_service.go`
- [Hooks](https://github.com/drmagice/gmqtt/blob/master/server/hook.go): extension points · Hook registration and event dispatch used by plugins and broker policies. · `server/hook.go`
- [API registrar](https://github.com/drmagice/gmqtt/blob/master/server/api_registrar.go): api wiring · Registers plugin-facing APIs onto the broker services. · `server/api_registrar.go`

### Persistence

Session, subscription, queue, unack, retained, and encoding backends.

- [Sessions](https://github.com/drmagice/gmqtt/blob/master/persistence/session/session.go): session store · Session abstraction with memory and Redis implementations. · `persistence/session/session.go`
- [Subscriptions](https://github.com/drmagice/gmqtt/blob/master/persistence/subscription/subscription.go): subscription index · Topic subscription storage and matching, including trie and Redis backends. · `persistence/subscription/subscription.go`
- [Queues](https://github.com/drmagice/gmqtt/blob/master/persistence/queue/queue.go): QoS queue · In-flight delivery queues and delivery ordering for QoS flows. · `persistence/queue/queue.go`
- [Unacked](https://github.com/drmagice/gmqtt/blob/master/persistence/unack/unack.go): ack tracker · Tracks acknowledgements and retry state for durable delivery semantics. · `persistence/unack/unack.go`
- [Retained](https://github.com/drmagice/gmqtt/blob/master/retained/interface.go): retained store · Retained-message storage and trie-based topic lookup. · `retained/interface.go`

### Plugins

Built-in extension modules and their generated API surfaces.

- [Auth plugin](https://github.com/drmagice/gmqtt/blob/master/plugin/auth/auth.go): auth extension · Username/password authentication and account management plugin. · `plugin/auth/auth.go`
- [Admin plugin](https://github.com/drmagice/gmqtt/blob/master/plugin/admin/admin.go): management api · Operational APIs over gRPC, REST gateway, and Swagger artifacts. · `plugin/admin/admin.go`
- [Prometheus](https://github.com/drmagice/gmqtt/blob/master/plugin/prometheus/prometheus.go): metrics plugin · Exports broker metrics for Prometheus scraping. · `plugin/prometheus/prometheus.go`
- [Federation](https://github.com/drmagice/gmqtt/blob/master/plugin/federation/federation.go): cluster plugin · Experimental inter-node membership and coordination layer. · `plugin/federation/federation.go`

### Shared packages

Protocol types, domain objects, and small shared utilities.

- [Config](https://github.com/drmagice/gmqtt/blob/master/config/config.go): runtime config · Broker, MQTT, persistence, API, and topic-alias configuration models. · `config/config.go`
- [Packets](https://github.com/drmagice/gmqtt/blob/master/pkg/packets/packets.go): mqtt codec · MQTT packet types and encoding/decoding for connect, publish, subscribe, ack, and auth flows. · `pkg/packets/packets.go`
- [Domain models](https://github.com/drmagice/gmqtt/blob/master/message.go): shared state · Message, session, and subscription abstractions shared across broker subsystems. · `message.go`
- [Topic alias](https://github.com/drmagice/gmqtt/blob/master/topicalias/fifo/fifo.go): aliasing · Topic-alias helpers used by publish routing. · `topicalias/fifo/fifo.go`

## Connections

- gmqttd → Config: loads · Broker daemon reads runtime configuration before starting services.
- gmqttd → Server: starts · Daemon boots the broker server and network listeners.
- gmqttd → Auth plugin: enables · Daemon composes built-in plugins into the broker process.
- gmqttd → Admin plugin: enables · Daemon composes built-in plugins into the broker process.
- gmqttd → Prometheus: enables · Daemon composes built-in plugins into the broker process.
- gmqttd → Federation: enables · Daemon composes built-in plugins into the broker process.
- gmqctl → Admin plugin: operates · CLI supports admin and plugin-related workflows.
- gmqctl → Auth plugin: operates · CLI supports admin and plugin-related workflows.
- Server → Clients: owns · Core server manages client lifecycle and connection state.
- Server → Publish path: routes · Core server drives publish fan-out and delivery coordination.
- Server → Hooks: dispatches · Core server emits lifecycle and message hooks.
- Server → API registrar: registers · Core server exposes plugin APIs through a registrar.
- Server → Sessions: uses · Server depends on abstract session storage.
- Server → Subscriptions: uses · Server depends on subscription matching and storage.
- Server → Queues: uses · Server relies on queue state for QoS delivery.
- Server → Unacked: uses · Server relies on acknowledgement tracking for retries.
- Server → Retained: uses · Server consults retained-message storage during routing.
- Publish path → Packets: parses · Publish flow consumes and emits MQTT packets.
- Publish path → Topic alias: applies · Publish pipeline resolves topic aliases.
- Clients → Packets: parses · Client I/O turns network bytes into protocol packets.
- Clients → Domain models: tracks · Client state uses shared message and session concepts.
- Sessions → Config: configured by · Session backend selection and options come from config.
- Subscriptions → Config: configured by · Subscription backend selection and options come from config.
- Queues → Config: configured by · Queue backend selection and options come from config.
- Unacked → Config: configured by · Ack-tracking backend selection and options come from config.
- Auth plugin → Hooks: hooks · Auth plugin plugs into connection and authentication events.
- Admin plugin → API registrar: registers · Admin plugin exposes APIs through the broker registrar.
- Prometheus → Hooks: hooks · Metrics plugin observes broker events via hooks.
- Federation → Hooks: hooks · Federation plugin participates through server hook events.
- Federation → API registrar: registers · Federation plugin exposes management or coordination APIs.
- Retained → Packets: stores · Retained messages are populated from publish packets.
- Server → Config: reads · Server behavior is governed by broker configuration.
