# bathtimefish/kraken_collector architecture

> Architecture diagram of the GitHub repository bathtimefish/kraken_collector, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bathtimefish/kraken_collector
- Repository: https://github.com/bathtimefish/kraken_collector

## Overview

##### Main Components
- Collector Core: Represented by the src/main.rs and src/service.rs files, this component serves as the entry point of the application. It ties together all collectors and handles the gRPC communication with Kraken Broker.
– Webhook Collector: Handles HTTP POST requests (triggered by environment variables KRKNC_WEBHOOK_PATH and KRKNC_WEBHOOK_PORT).
– MQTT Collector: This module uses configuration from both environment variables (KRKNC_MQTT_HOST, KRKNC_MQTT_TOPIC, KRKNC_MQTT_CONFIG_PATH) and the configuration file (config/rumqttd.toml). It leverages the rumqttd library.
– Websocket Collector: Listens for WebSocket connections as specified via KRKNC_WEBSOCKET_HOST and KRKNC_WEBSOCKET_SUB_PROTOCOL.
– Additional Collectors: There are other protocol collectors (for example, grpc, ibeacon, serial) found in the collectors subdirectory, suggesting that the system is extensible.
- gRPC Client Interface: The proto/kraken.proto file and parts of the code facilitate the gRPC communication to send collected data to Kraken Broker.
- Configuration & Environment Management: The assets and config directories (with Dockerfile, docker-compose.yml, and other environment setup scripts) define deployment, runtime settings, and allowed parameters.
- Test Tools: The test-tools directory includes various test components (mock MQTT broker, gRPC server, websocket client/server) which help simulate and validate communication between components.

##### Relationships and Interactions
- Incoming Data: Edge IoT sensors send data via HTTP, MQTT, or Websocket. Each collector (webhook, MQTT, websocket) listens on its designated host/port.
- Internal Routing: Once a collector component receives data, it passes the payload to the Collector Core. The core (service.rs) then processes or forwards the data.
- Broker Communication: The Collector Core acts as a gRPC client as defined by the proto file and connects to the Kraken Broker (as set by KRKNC_BROKER_HOST). This is a clear interface point that connects the collection layer with downstream processing.
- Extensibility: The modular design (with separate modules for each protocol in src/collectors) highlights an architectural pattern that allows additional protocols or collectors to be plugged in with minimal changes.

##### Architectural Patterns and Design Principles
- Modular Design/Separation of Concerns: Different protocol handlers are isolated into their own modules which simplifies testing and future expansion.
- Configuration-Driven Behavior: The use of environment variables and configuration files (e.g., rumqttd.toml, allowed_uuids.yml) suggests that the functionality is easily configurable, allowing the same core code to work in multiple deployment scenarios (cloud vs. on-premise).
- Client-Server Communication: The use of gRPC for connecting to the external Kraken Broker forms a clear client-server boundary within the overall IoT system.
- Extensibility: The structure clearly allows one to extend collector functionality by adding new modules under src/collectors.

##### Relevant Technologies and Tools
- Rust: The primary development language.
- gRPC/protobuf: For communication with Kraken Broker.
- MQTT (via rumqttd): Used as the messaging protocol for one of the collectors.
- Web Technologies: HTTP/webhooks and Websocket protocols are supported.
- Docker and related assets: Enable containerized deployment and test environments.

##### Elements
- Clear Labels for Components: “Collector Core (main.rs/service.rs)”, “Webhook Collector”, “MQTT Collector”, “Websocket Collector”, “gRPC Client/Bridge to Kraken Broker”, “Configuration & Deployment Scripts (Dockerfile, env scripts)”, and “Test Tools”.
- Annotations: For extra clarity, label environment variables and protocol-specific ports as indicated in the README (e.g., KRKNC_WEBHOOK_PORT, KRKNC_MQTT_HOST, etc.).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Edge IoT Sensors
    subgraph "Edge IoT Sensors"
        Sensors["Edge IoT Sensors"]:::sensors
    end

    %% Protocol-Specific Collectors
    subgraph "Protocol-Specific Collectors"
        Webhook["Webhook Collector (KRKNC_WEBHOOK_PORT)"]:::collector
        MQTT["MQTT Collector (KRKNC_MQTT_HOST)"]:::collector
        WS["Websocket Collector (KRKNC_WEBSOCKET_HOST)"]:::collector
        %% Additional Collectors Subgraph
        subgraph "Additional Collectors"
            IBeacon["iBeacon Collector"]:::collector
            Serial["Serial Collector"]:::collector
        end
        MQTTConfig["MQTT Config (rumqttd.toml)"]:::configExtra
    end

    %% Collector Core Layer
    subgraph "Collector Core"
        CoreMain["Core - main.rs"]:::core
        CoreService["Core - service.rs"]:::core
    end

    %% gRPC Client Interface
    subgraph "gRPC Client Interface"
        GRPCProto["gRPC Proto Interface"]:::grpc
        GRPCColl["gRPC Collector"]:::grpc
    end

    %% External Broker
    KrakenBroker["Kraken Broker (gRPC)"]:::external

    %% Configuration & Environment
    Configuration["Configuration & Env"]:::config

    %% Test Tools
    TestTools["Test Tools"]:::test

    %% Data Flow from Sensors to Collectors
    Sensors -->|"HTTP/MQTT/Websocket"| Webhook
    Sensors -->|"HTTP/MQTT/Websocket"| MQTT
    Sensors -->|"HTTP/MQTT/Websocket"| WS
    Sensors -->|"Other Protocol Data"| IBeacon
    Sensors -->|"Other Protocol Data"| Serial

    %% Connectors: Collectors to Collector Core
    Webhook -->|"data"| CoreMain
    MQTT -->|"data"| CoreMain
    WS -->|"data"| CoreMain
    IBeacon -->|"data"| CoreMain
    Serial -->|"data"| CoreMain

    %% MQTT Collector dependency on its configuration file
    MQTTConfig ---|"config"| MQTT

    %% Collector Core to gRPC Client Interface
    CoreService -->|"forward data"| GRPCColl
    %% Internal connection within gRPC Interface (from proto definition to collector)
    GRPCProto --- GRPCColl
    %% gRPC Client Interface to Broker
    GRPCColl -->|"gRPC call"| KrakenBroker

    %% Configuration dependency arrows (dotted)
    Configuration ---|"env/config"| Webhook
    Configuration ---|"env/config"| MQTT
    Configuration ---|"env/config"| WS
    Configuration ---|"env/config"| CoreMain
    Configuration ---|"env/config"| CoreService
    Configuration ---|"env/config"| GRPCProto
    Configuration ---|"env/config"| GRPCColl

    %% Test Tools connection (used to simulate/test various components)
    TestTools ---|"simulate"| KrakenBroker

    %% Click Events for Components
    click CoreMain "https://github.com/bathtimefish/kraken_collector/blob/main/src/main.rs"
    click CoreService "https://github.com/bathtimefish/kraken_collector/blob/main/src/service.rs"
    click Webhook "https://github.com/bathtimefish/kraken_collector/blob/main/src/collectors/webhook.rs"
    click MQTT "https://github.com/bathtimefish/kraken_collector/blob/main/src/collectors/mqtt.rs"
    click MQTTConfig "https://github.com/bathtimefish/kraken_collector/blob/main/config/rumqttd.toml"
    click WS "https://github.com/bathtimefish/kraken_collector/blob/main/src/collectors/websocket.rs"
    click GRPCProto "https://github.com/bathtimefish/kraken_collector/blob/main/proto/kraken.proto"
    click GRPCColl "https://github.com/bathtimefish/kraken_collector/blob/main/src/collectors/grpc.rs"
    click IBeacon "https://github.com/bathtimefish/kraken_collector/blob/main/src/collectors/ibeacon.rs"
    click Serial "https://github.com/bathtimefish/kraken_collector/blob/main/src/collectors/serial.rs"
    click Configuration "https://github.com/bathtimefish/kraken_collector/tree/main/assets"
    click TestTools "https://github.com/bathtimefish/kraken_collector/tree/main/test-tools"

    %% Styling Classes
    classDef sensors fill:#ffcccc,stroke:#cc0000,stroke-width:2px;
    classDef collector fill:#ccffcc,stroke:#006600,stroke-width:2px;
    classDef core fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef grpc fill:#ffeeba,stroke:#856404,stroke-width:2px;
    classDef external fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef config fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef test fill:#e2e3e5,stroke:#d6d8db,stroke-width:2px;
    classDef configExtra fill:#fdfd96,stroke:#999900,stroke-width:2px;
```
