# thingsboard/thingsboard-gateway architecture

> Architecture diagram of the GitHub repository thingsboard/thingsboard-gateway, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thingsboard/thingsboard-gateway
- Repository: https://github.com/thingsboard/thingsboard-gateway

## Overview

##### Project Type and Purpose
- This project is an open-source IoT gateway built in Python. Its purpose is to bridge devices and various protocols (OPC-UA, MQTT, Modbus, BLE, REST, etc.) to the ThingsBoard IoT platform.
- The gateway supports custom connectors and converters. It is designed using a modular, microservices-like approach (though within one Python process) that enables extensibility and separation of concerns, notably in handling device integration, data mapping, persistence, and remote communications.

##### Main Structural Components (as derived from the file tree and README)
- Core Gateway Engine: The central “brains” of the system found in the thingsboard_gateway/ directory. This includes:
- The gateway module (thingsboard_gateway/gateway) that handles overall configuration, device provisioning, and message routing.
- The configuration sub-module (thingsboard_gateway/config) containing various JSON files for different protocols.
- Persistence and storage components in thingsboard_gateway/storage that guarantee data delivery on failure.
- The tb_utility module which provides operational support, logging, remote updates, and a hot-reloader.
- Connector Modules: Each protocol (OPC-UA, MQTT, Modbus, BLE, CAN, BACnet, SNMP, ODBC, FTP, Socket, XMPP, OCPP, and custom connectors) has its own module within thingsboard_gateway/connectors. They initiate data collection from various sources, process the input (via dedicated converters), and interface with the core engine.
- Converter Components: Present across connectors, often implemented as separate converter modules (for example in thingsboard_gateway/connectors/modbus regarding modbus converters) to translate device-specific data into a unified format.
- gRPC and External Communication: The project also includes gRPC connectors (in thingsboard_gateway/grpc_connectors) for remote procedure calls, as well as REST and other API-based integrations.
- Extensions: The thingsboard_gateway/extensions directory provides additional custom integrations and backward compatibility support.
- Testing and Deployment: With a comprehensive tests folder (unit, integration, blackbox) and Docker-related files, the project demonstrates solid CI/CD practices and containerization for deployment.

##### Core Gateway Engine Details
- The core engine is responsible for:
- Managing and orchestrating the connectors.
- Processing configuration data to initialize connectors.
- Routing data messages through converters before forwarding them to the ThingsBoard cluster.
- Maintaining local persistence (file, SQLite, memory) to ensure data integrity in the case of network or hardware failures.
- Supporting remote operations such as logging, RPC methods (for controlling the gateway from the ThingsBoard UI), and a remote shell for system management.
- It implements a flexible plug-in architecture for both connectors and converters, allowing dynamic loading of new modules.
- The engine also handles error management and automatic reconnection to the ThingsBoard cluster.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Connector Modules Subgraph
    subgraph "Connector Modules"
        OPCUA["OPC-UA Connector"]:::connector
        MQTT["MQTT Connector"]:::connector
        MODBUS["Modbus Connector"]:::connector
        BLE["BLE Connector"]:::connector
    end

    %% Converter Components Subgraph
    subgraph "Converter Components"
        GenericConv["Generic Converter"]:::converter
        ModbusConv["Modbus Uplink Converter"]:::converter
        MQTTConv["MQTT Uplink Converter"]:::converter
    end

    %% Configuration and Utility Subgraph
    subgraph "Configuration and Utility"
        Config["Configuration Files"]:::config
        Utility["TB Utility Tools"]:::config
    end

    %% Core Gateway Engine Subgraph with Detailed Functions
    subgraph "Core Gateway Engine"
        CGE["Core Gateway Engine"]:::engine
        subgraph "Engine Functions"
            CP["Configuration Processing"]:::engineDetail
            DP["Device Provisioning"]:::engineDetail
            MR["Message Routing & Converter Registration"]:::engineDetail
            EM["Error Handling & Auto-Reconnect"]:::engineDetail
            RM["Remote Management (RPC/Shell)"]:::engineDetail
            CP --> DP
            DP --> MR
            MR --> EM
            EM --> RM
        end
    end

    %% Other System Components
    PERSIST["Persistence & Storage"]:::storage
    GRPC["gRPC & External Communication"]:::external
    EXT["Extensions"]:::external
    TEST["Testing & Deployment"]:::external

    %% Relationships
    %% Connectors flow into Core Gateway Engine (some via converters)
    OPCUA -->|"sends data"| CGE
    BLE -->|"sends data"| CGE

    MQTT -->|"converts data"| MQTTConv
    MQTTConv -->|"sends converted data"| CGE

    MODBUS -->|"converts data"| ModbusConv
    ModbusConv -->|"sends converted data"| CGE

    %% Generic converter as an alternative path (if used)
    GenericConv -->|"converts generic data"| CGE

    %% Configuration and Utility provide initialization to Core Engine
    Config -->|"loads settings"| CGE
    Utility -->|"supports operations"| CGE

    %% Core Engine outputs data and operations to other layers
    CGE -->|"persists data"| PERSIST
    CGE -->|"communicates via"| GRPC
    CGE -->|"loads extensions"| EXT
    CGE -->|"undergoes tests/deployment"| TEST

    %% Click Events for Components
    click CGE "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/gateway"
    click Config "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/config"
    click Utility "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/tb_utility"
    click PERSIST "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/storage"
    click OPCUA "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/connectors/opcua"
    click MQTT "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/connectors/mqtt"
    click MODBUS "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/connectors/modbus"
    click BLE "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/connectors/ble"
    click GenericConv "https://github.com/thingsboard/thingsboard-gateway/blob/master/thingsboard_gateway/connectors/converter.py"
    click ModbusConv "https://github.com/thingsboard/thingsboard-gateway/blob/master/thingsboard_gateway/connectors/modbus/bytes_modbus_uplink_converter.py"
    click MQTTConv "https://github.com/thingsboard/thingsboard-gateway/blob/master/thingsboard_gateway/connectors/mqtt/mqtt_uplink_converter.py"
    click GRPC "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/grpc_connectors"
    click EXT "https://github.com/thingsboard/thingsboard-gateway/tree/master/thingsboard_gateway/extensions"
    click TEST "https://github.com/thingsboard/thingsboard-gateway/tree/master/tests"

    %% Styles
    classDef engine fill:#F08080,stroke:#333,stroke-width:2px;
    classDef engineDetail fill:#FFB6C1,stroke:#333,stroke-width:1px;
    classDef connector fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef converter fill:#FFC0CB,stroke:#333,stroke-width:2px;
    classDef config fill:#FAFAD2,stroke:#333,stroke-width:2px;
    classDef storage fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:2px;
```
