How bathtimefish/kraken_collector works
Main Components
• Collector Core: Represented by the src/main.rs and src/service.rs files, this component serves as the entry point of the application. It ties together all collectors and handles the gRPC communication with Kraken Broker.
– Webhook Collector: Handles HTTP POST requests (triggered by environment variables KRKNC_WEBHOOK_PATH and KRKNC_WEBHOOK_PORT).
– MQTT Collector: This module uses configuration from both environment variables (KRKNC_MQTT_HOST, KRKNC_MQTT_TOPIC, KRKNC_MQTT_CONFIG_PATH) and the configuration file (config/rumqttd.toml). It leverages the rumqttd library.
– Websocket Collector: Listens for WebSocket connections as specified via KRKNC_WEBSOCKET_HOST and KRKNC_WEBSOCKET_SUB_PROTOCOL.
– Additional Collectors: There are other protocol collectors (for example, grpc, ibeacon, serial) found in the collectors subdirectory, suggesting that the system is extensible.
• gRPC Client Interface: The proto/kraken.proto file and parts of the code facilitate the gRPC communication to send collected data to Kraken Broker.
• Configuration & Environment Management: The assets and config directories (with Dockerfile, docker-compose.yml, and other environment setup scripts) define deployment, runtime settings, and allowed parameters.
• Test Tools: The test-tools directory includes various test components (mock MQTT broker, gRPC server, websocket client/server) which help simulate and validate communication between components.
Relationships and Interactions
• Incoming Data: Edge IoT sensors send data via HTTP, MQTT, or Websocket. Each collector (webhook, MQTT, websocket) listens on its designated host/port.
• Internal Routing: Once a collector component receives data, it passes the payload to the Collector Core. The core (service.rs) then processes or forwards the data.
• Broker Communication: The Collector Core acts as a gRPC client as defined by the proto file and connects to the Kraken Broker (as set by KRKNC_BROKER_HOST). This is a clear interface point that connects the collection layer with downstream processing.
• Extensibility: The modular design (with separate modules for each protocol in src/collectors) highlights an architectural pattern that allows additional protocols or collectors to be plugged in with minimal changes.
Architectural Patterns and Design Principles
• Modular Design/Separation of Concerns: Different protocol handlers are isolated into their own modules which simplifies testing and future expansion.
• Configuration-Driven Behavior: The use of environment variables and configuration files (e.g., rumqttd.toml, allowed_uuids.yml) suggests that the functionality is easily configurable, allowing the same core code to work in multiple deployment scenarios (cloud vs. on-premise).
• Client-Server Communication: The use of gRPC for connecting to the external Kraken Broker forms a clear client-server boundary within the overall IoT system.
• Extensibility: The structure clearly allows one to extend collector functionality by adding new modules under src/collectors.
Relevant Technologies and Tools
• Rust: The primary development language.
• gRPC/protobuf: For communication with Kraken Broker.
• MQTT (via rumqttd): Used as the messaging protocol for one of the collectors.
• Web Technologies: HTTP/webhooks and Websocket protocols are supported.
• Docker and related assets: Enable containerized deployment and test environments.
Elements
• Clear Labels for Components: “Collector Core (main.rs/service.rs)”, “Webhook Collector”, “MQTT Collector”, “Websocket Collector”, “gRPC Client/Bridge to Kraken Broker”, “Configuration & Deployment Scripts (Dockerfile, env scripts)”, and “Test Tools”.
• Annotations: For extra clarity, label environment variables and protocol-specific ports as indicated in the README (e.g., KRKNC_WEBHOOK_PORT, KRKNC_MQTT_HOST, etc.).