# kuldeepsinh/yams architecture

> Architecture diagram of the GitHub repository kuldeepsinh/yams, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kuldeepsinh/yams
- Repository: https://github.com/kuldeepsinh/yams

## Overview

##### Project type and purpose
- This is an Erlang/OTP application implementing an MQTT broker (“Yet Another MQTT Server”).
- Its goal is to accept thousands/millions of concurrent TCP connections from MQTT clients, route publish/subscribe messages, manage topic state, and optionally persist data.

##### File structure
- Top-level files:
- rebar.config (build configuration)
- /yams_db.hrl (shared record/type definitions)
- README.md, LICENSE, NOTICE
- Sandbox directory: lightweight demos/tests of core modules (coner, pinger, puber, router, suber).
- src directory: the OTP application code.
- yams.app.src, yams_app.erl, yams_sup.erl – application entry point and top supervisor
- * _sup.erl files – supervisors for each major subsystem
- Erlang modules handling:
– TCP Listener & Acceptor (listener.erl, acceptor.erl)
– Connection/session management (connect.erl, client_to_apid.erl)
– Message dispatch & relay (publish.erl, subscribe.erl, router.erl)
– Topic parsing (topic_parser.erl), subscriptions DB (subscriptions.erl), publications DB (publications.erl), and the yams_db interface

##### Key insights from README
- Plain description: it’s a standalone MQTT broker written to leverage Erlang/OTP concurrency & supervision.

a. Main components
1. External MQTT Clients (any MQTT‐compatible client)
2. Network Stack
– Listener (listener.erl)
– Acceptor (acceptor.erl)
3. Connection Manager
– Connection processes (connect.erl)
– Client-to-AppID translator (client_to_apid.erl)
4. Message Handling
– Publisher (publish.erl) & Subscription (subscribe.erl) interfaces
– Router (router.erl) for matching topics to subscribers
– Topic Parser (topic_parser.erl)
5. State Storage
– Subscriptions datastore (subscriptions.erl)
– Publications datastore (publications.erl)
– yams_db interface (yams_db.erl + include)
6. OTP Supervision Tree
– Top‐level: yams_app → yams_sup
– Child supervisors: listener_sup, acceptor_sup, connect_sup, publish_sup, subscribe_sup, router_sup, topic_parser_sup, mh_sup, msg_sup
7. Utility library (yams_lib.erl)

b. Relationships and interactions
- Clients → TCP Listener (port) → Acceptor spawns a Connection →
Connection process handles handshake/auth →
Based on MQTT packet type, forward to Publish or Subscribe subsystem →
Router & Topic Parser dispatch the message →
Update Subscriptions or Publications datastore →
Router notifies all matching Connection processes →
Connections push packets back to clients.
- Supervisors monitor sets of workers for fault ­tolerance.

c. Architectural patterns & principles
- OTP supervision tree for process isolation and automated restarts.
- “One connection = one process” concurrency model.
- Clear separation of concerns: networking vs message logic vs storage.

d. Technologies / frameworks
- Erlang/OTP (gen_server, supervisor behaviors)
- rebar (build tool)
- ETS or Mnesia (potential backing store inside yams_db)

##### Capturing essential elements
- Don’t omit any OTP supervisor or major worker: each _sup and its child group should appear.
- Yams_db.hrl as the schema interface for the datastores.
- If desired, in a corner show the sandbox folder as “Test Harnesses” separate from the production supervision tree.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Entity
    Clients["External MQTT Clients"]:::external

    %% Supervision Tree
    subgraph "OTP Supervision Tree"
        direction TB
        YAMS_APP["yams_app"]:::supervisor
        YAMS_SUP["yams_sup"]:::supervisor
        YAMS_APPSRC["yams.app.src"]:::config

        YAMS_APP --> YAMS_SUP
        YAMS_APP --> YAMS_APPSRC

        subgraph "Child Supervisors"
            direction TB
            LIST_SUP["listener_sup"]:::supervisor
            ACC_SUP["acceptor_sup"]:::supervisor
            CONN_SUP["connect_sup"]:::supervisor
            PUB_SUP["publish_sup"]:::supervisor
            SUB_SUP["subscribe_sup"]:::supervisor
            ROUTE_SUP["router_sup"]:::supervisor
            TP_SUP["topic_parser_sup"]:::supervisor
            MH_SUP["mh_sup"]:::supervisor
            MSG_SUP["msg_sup"]:::supervisor
        end

        YAMS_SUP --> LIST_SUP
        YAMS_SUP --> ACC_SUP
        YAMS_SUP --> CONN_SUP
        YAMS_SUP --> PUB_SUP
        YAMS_SUP --> SUB_SUP
        YAMS_SUP --> ROUTE_SUP
        YAMS_SUP --> TP_SUP
        YAMS_SUP --> MH_SUP
        YAMS_SUP --> MSG_SUP
    end

    %% Network Stack
    subgraph "Network Stack"
        direction TB
        Listener["listener"]:::worker
        Acceptor["acceptor"]:::worker
    end

    LIST_SUP --> Listener
    ACC_SUP --> Acceptor

    %% Connection Manager
    subgraph "Connection Manager"
        direction TB
        Connect["connect"]:::worker
        ClientToAppID["client_to_apid"]:::worker
    end

    CONN_SUP --> Connect
    CONN_SUP --> ClientToAppID

    %% Message Handling
    subgraph "Message Handling"
        direction TB
        Publish["publish"]:::worker
        Subscribe["subscribe"]:::worker
        Router["router"]:::worker
        TopicParser["topic_parser"]:::worker
    end

    PUB_SUP --> Publish
    SUB_SUP --> Subscribe
    ROUTE_SUP --> Router
    TP_SUP --> TopicParser

    %% State Storage
    subgraph "State Storage"
        direction TB
        SubscriptionsDB["subscriptions"]:::datastore
        PublicationsDB["publications"]:::datastore
        YamsDB["yams_db"]:::datastore
        YamsDBHrl["yams_db.hrl"]:::datastore
    end

    %% Utility Library
    YamsLib["yams_lib"]:::worker

    %% Sandbox / Test Harnesses
    Sandbox["sandbox/ (Test Harnesses)"]:::external

    %% Data and Control Flows
    Clients -->|CONNECT| Listener
    Listener -->|spawn| Acceptor
    Acceptor -->|establish session| Connect
    Connect -->|translate| ClientToAppID
    ClientToAppID -->|packet type| Publish
    ClientToAppID -->|packet type| Subscribe
    Publish -->|PUBLISH| Router
    Subscribe -->|SUBSCRIBE| Router
    Router -->|parse topics| TopicParser
    TopicParser -->|update| SubscriptionsDB
    TopicParser -->|update| PublicationsDB
    SubscriptionsDB -->|lookup| Router
    PublicationsDB -->|lookup| Router
    Router -->|dispatch| Connect
    Connect -->|publish to| Clients

    %% Legend
    subgraph Legend
        direction TB
        legendExt["Grey = External"]:::external
        legendSup["Green = Supervisor"]:::supervisor
        legendWork["Blue = Worker"]:::worker
        legendData["Orange = Datastore"]:::datastore
        legendCfg["Purple = Config"]:::config
    end

    %% Click Events
    click Listener "https://github.com/kuldeepsinh/yams/blob/master/src/listener.erl"
    click LIST_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/listener_sup.erl"
    click Acceptor "https://github.com/kuldeepsinh/yams/blob/master/src/acceptor.erl"
    click ACC_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/acceptor_sup.erl"
    click Connect "https://github.com/kuldeepsinh/yams/blob/master/src/connect.erl"
    click CONN_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/connect_sup.erl"
    click ClientToAppID "https://github.com/kuldeepsinh/yams/blob/master/src/client_to_apid.erl"
    click Publish "https://github.com/kuldeepsinh/yams/blob/master/src/publish.erl"
    click PUB_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/publish_sup.erl"
    click Subscribe "https://github.com/kuldeepsinh/yams/blob/master/src/subscribe.erl"
    click SUB_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/subscribe_sup.erl"
    click Router "https://github.com/kuldeepsinh/yams/blob/master/src/router.erl"
    click ROUTE_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/router_sup.erl"
    click TopicParser "https://github.com/kuldeepsinh/yams/blob/master/src/topic_parser.erl"
    click TP_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/topic_parser_sup.erl"
    click SubscriptionsDB "https://github.com/kuldeepsinh/yams/blob/master/src/subscriptions.erl"
    click PublicationsDB "https://github.com/kuldeepsinh/yams/blob/master/src/publications.erl"
    click YamsDB "https://github.com/kuldeepsinh/yams/blob/master/src/yams_db.erl"
    click YamsDBHrl "https://github.com/kuldeepsinh/yams/blob/master/include/yams_db.hrl"
    click YAMS_APP "https://github.com/kuldeepsinh/yams/blob/master/src/yams_app.erl"
    click YAMS_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/yams_sup.erl"
    click YAMS_APPSRC "https://github.com/kuldeepsinh/yams/blob/master/src/yams.app.src"
    click MH_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/mh_sup.erl"
    click MSG_SUP "https://github.com/kuldeepsinh/yams/blob/master/src/msg_sup.erl"
    click YamsLib "https://github.com/kuldeepsinh/yams/blob/master/src/yams_lib.erl"
    click Sandbox "https://github.com/kuldeepsinh/yams/tree/master/sandbox/"

    %% Styles
    classDef supervisor fill:#b7e3a8,stroke:#4a653d,color:#000
    classDef worker fill:#a8d0e3,stroke:#25607b,color:#000
    classDef datastore fill:#f5c27f,stroke:#a46a00,color:#000
    classDef external fill:#d3d3d3,stroke:#7a7a7a,color:#000
    classDef config fill:#d3a8e3,stroke:#67298a,color:#000
```
