How thingsboard/thingsboard-gateway works
Project Type and Purpose
• This project is an open-source IoT gateway built in Python. Its purpose is to bridge devices and various protocols (OPC-UA, MQTT, Modbus, BLE, REST, etc.) to the ThingsBoard IoT platform.
• The gateway supports custom connectors and converters. It is designed using a modular, microservices-like approach (though within one Python process) that enables extensibility and separation of concerns, notably in handling device integration, data mapping, persistence, and remote communications.
Main Structural Components (as derived from the file tree and README)
• Core Gateway Engine: The central “brains” of the system found in the thingsboard_gateway/ directory. This includes:
• The gateway module (thingsboard_gateway/gateway) that handles overall configuration, device provisioning, and message routing.
• The configuration sub-module (thingsboard_gateway/config) containing various JSON files for different protocols.
• Persistence and storage components in thingsboard_gateway/storage that guarantee data delivery on failure.
• The tb_utility module which provides operational support, logging, remote updates, and a hot-reloader.
• Connector Modules: Each protocol (OPC-UA, MQTT, Modbus, BLE, CAN, BACnet, SNMP, ODBC, FTP, Socket, XMPP, OCPP, and custom connectors) has its own module within thingsboard_gateway/connectors. They initiate data collection from various sources, process the input (via dedicated converters), and interface with the core engine.
• Converter Components: Present across connectors, often implemented as separate converter modules (for example in thingsboard_gateway/connectors/modbus regarding modbus converters) to translate device-specific data into a unified format.
• gRPC and External Communication: The project also includes gRPC connectors (in thingsboard_gateway/grpc_connectors) for remote procedure calls, as well as REST and other API-based integrations.
• Extensions: The thingsboard_gateway/extensions directory provides additional custom integrations and backward compatibility support.
• Testing and Deployment: With a comprehensive tests folder (unit, integration, blackbox) and Docker-related files, the project demonstrates solid CI/CD practices and containerization for deployment.
Core Gateway Engine Details
• The core engine is responsible for:
• Managing and orchestrating the connectors.
• Processing configuration data to initialize connectors.
• Routing data messages through converters before forwarding them to the ThingsBoard cluster.
• Maintaining local persistence (file, SQLite, memory) to ensure data integrity in the case of network or hardware failures.
• Supporting remote operations such as logging, RPC methods (for controlling the gateway from the ThingsBoard UI), and a remote shell for system management.
• It implements a flexible plug-in architecture for both connectors and converters, allowing dynamic loading of new modules.
• The engine also handles error management and automatic reconnection to the ThingsBoard cluster.