Try   HackMD

TB w/ Shelly EM Integration

Shelly Gen 1 設備,感測端回傳數值分別有各自相對應 topic,連同感測數值,不符合 TB telemetry JSON 格式 (key: value)。

When configured for MQTT Shelly EM reports data on:

  • shellies/shellyem-<deviceid>/emeter/<i>/pf power factor
  • shellies/shellyem-<deviceid>/emeter/<i>/power instantaneous active power in Watts
  • shellies/shellyem-<deviceid>/emeter/<i>/reactive_power instantaneous reactive power in Watts
  • shellies/shellyem-<deviceid>/emeter/<i>/total total energy in Wh (accumulated in device's non-volatile memory)
  • shellies/shellyem-<deviceid>/emeter/<i>/total_returned total energy returned to the grid in Wh (accumulated in device's non-volatile memory)
  • shellies/shellyem-<deviceid>/emeter/<i>/voltage grid voltage in Volts

要讓 Shelly EM,透過 MQTT,回傳感測數值至 TB,其一解法詳見本文。
TB <> TB IoT Gateway (ShellyEMMqttUplinkConverter) <> Mosquitto Broker <> Device: Shelly EM

ThingsBoard

  • Dashbaords > Gateways > Add device
  • Devices > Shelly-GW > Copy access token

ThingsBoard IoT Gateway w/ Mosquitto Mqtt Broker

Mosquitto mqtt broker

Package Installation
sudo apt install mosquitto
$ sudo systemctl status mosquitto.service
● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
     Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-08-08 02:15:31 UTC; 12s ago
       Docs: man:mosquitto.conf(5)
             man:mosquitto(8)
   Main PID: 1512271 (mosquitto)
      Tasks: 3 (limit: 19098)
     Memory: 1.4M
     CGroup: /system.slice/mosquitto.service
             └─1512271 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

Aug 08 02:15:31 sw systemd[1]: Starting Mosquitto MQTT v3.1/v3.1.1 Broker...
Aug 08 02:15:31 sw mosquitto[1512271]: [3616570.595474]~DLT~1512271~INFO     ~FIFO /tmp/dlt cannot be opened. Retrying late>
Aug 08 02:15:31 sw systemd[1]: Started Mosquitto MQTT v3.1/v3.1.1 Broker.

Config and Debug

  • Config files: /etc/mosquitto/mosquitto.conf (/etc/mosquitto/cond.d/local.conf)
    more
    ​​​​sudo wget https://github.com/eclipse/mosquitto/blob/master/mosquitto.conf -O /etc/mosquitto/conf.d/local.conf
    ​​​​
    ​​​​sudo vi /etc/mosquitto/conf.d/local.conf
    ​​​​+line 233
    ​​​​listener 1883
    ​​​​+line 531
    ​​​​allow_anonymous true
    
  • Log files: /var/log/mosquitto/mosquitto.log

ThingsBoard IoT Gateway

Package Installation

  1. Download the deb file
    wget https://github.com/thingsboard/thingsboard-gateway/releases/latest/download/python3-thingsboard-gateway.deb
  2. Install the gateway using apt
    ​​​​sudo apt install ./python3-thingsboard-gateway.deb -y
    
    ​​​​$ pip3 show thingsboard-gateway
    ​​​​Name: thingsboard-gateway
    ​​​​Version: 3.1
    ​​​​Summary: Thingsboard Gateway for IoT devices.
    ​​​​Home-page: https://github.com/thingsboard/thingsboard-gateway
    ​​​​Author: ThingsBoard
    ​​​​Author-email: info@thingsboard.io
    ​​​​License: Apache Software License (Apache Software License 2.0)
    ​​​​Location: /usr/lib/python3/dist-packages
    ​​​​Requires: PyInquirer
    ​​​​Required-by:
    
  3. Check gateway status
  • Failed to start ThingsBoard Gateway
    ​​​​star@sw:~/workspace$ sudo systemctl status thingsboard-gateway.service
    ​​​​● thingsboard-gateway.service - ThingsBoard Gateway
    ​​​​     Loaded: loaded (/etc/systemd/system/thingsboard-gateway.service; enabled; vendor preset: enabled)
    ​​​​     Active: failed (Result: exit-code) since Mon 2022-08-08 08:34:28 UTC; 14min ago
    ​​​​    Process: 1527100 ExecStart=/usr/bin/python3 -c from thingsboard_gateway.tb_gateway import daemon; daemon() (code=exited, status=1/FAILURE)
    ​​​​   Main PID: 1527100 (code=exited, status=1/FAILURE)
    
    ​​​​Aug 08 08:34:28 sw systemd[1]: thingsboard-gateway.service: Scheduled restart job, restart counter is at 5.
    ​​​​Aug 08 08:34:28 sw systemd[1]: Stopped ThingsBoard Gateway.
    ​​​​Aug 08 08:34:28 sw systemd[1]: thingsboard-gateway.service: Start request repeated too quickly.
    ​​​​Aug 08 08:34:28 sw systemd[1]: thingsboard-gateway.service: Failed with result 'exit-code'.
    ​​​​Aug 08 08:34:28 sw systemd[1]: Failed to start ThingsBoard Gateway.
    
  • Check and upgrade protobuf version Active: active (running)
    terminal details
    ​​​​star@sw:~/workspace$ sudo pip3 show protobuf
    ​​​​Name: protobuf
    ​​​​Version: 3.6.1
    ​​​​Summary: Protocol Buffers
    ​​​​Home-page: https://developers.google.com/protocol-buffers/
    ​​​​Author: None
    ​​​​Author-email: None
    ​​​​License: 3-Clause BSD License
    ​​​​Location: /usr/lib/python3/dist-packages
    ​​​​Requires:
    ​​​​Required-by:
    
    ​​​​star@sw:~/workspace$ sudo pip3 install protobuf==3.20.1
    ​​​​Collecting protobuf==3.20.1
    ​​​​  Downloading protobuf-3.20.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
    ​​​​     |████████████████████████████████| 1.0 MB 901 kB/s
    ​​​​Installing collected packages: protobuf
    ​​​​  Attempting uninstall: protobuf
    ​​​​    Found existing installation: protobuf 3.6.1
    ​​​​    Not uninstalling protobuf at /usr/lib/python3/dist-packages, outside environment /usr
    ​​​​    Can't uninstall 'protobuf'. No files were found to uninstall.
    ​​​​Successfully installed protobuf-3.20.1
    ​​​​star@sw:~/workspace$ pip3 show protobuf
    ​​​​Name: protobuf
    ​​​​Version: 3.20.1
    ​​​​Summary: Protocol Buffers
    ​​​​Home-page: https://developers.google.com/protocol-buffers/
    ​​​​Author: None
    ​​​​Author-email: None
    ​​​​License: BSD-3-Clause
    ​​​​Location: /usr/local/lib/python3.8/dist-packages
    ​​​​Requires:
    ​​​​Required-by:
    ​​​​star@sw:~/workspace$ sudo systemctl status thingsboard-gateway.service
    ​​​​● thingsboard-gateway.service - ThingsBoard Gateway
    ​​​​     Loaded: loaded (/etc/systemd/system/thingsboard-gateway.service; enabled; vendor preset: enabled)
    ​​​​     Active: failed (Result: exit-code) since Mon 2022-08-08 08:34:28 UTC; 25min ago
    ​​​​    Process: 1527100 ExecStart=/usr/bin/python3 -c from thingsboard_gateway.tb_gateway import daemon; daemon() (code=exited, status=1/FAILURE)
    ​​​​   Main PID: 1527100 (code=exited, status=1/FAILURE)
    
    ​​​​Aug 08 08:34:28 sw systemd[1]: thingsboard-gateway.service: Scheduled restart job, restart counter is at 5.
    ​​​​Aug 08 08:34:28 sw systemd[1]: Stopped ThingsBoard Gateway.
    ​​​​Aug 08 08:34:28 sw systemd[1]: thingsboard-gateway.service: Start request repeated too quickly.
    ​​​​Aug 08 08:34:28 sw systemd[1]: thingsboard-gateway.service: Failed with result 'exit-code'.
    ​​​​Aug 08 08:34:28 sw systemd[1]: Failed to start ThingsBoard Gateway.
    ​​​​star@sw:~/workspace$ sudo systemctl start thingsboard-gateway.service
    ​​​​star@sw:~/workspace$ sudo systemctl status thingsboard-gateway.service
    ​​​​● thingsboard-gateway.service - ThingsBoard Gateway
    ​​​​     Loaded: loaded (/etc/systemd/system/thingsboard-gateway.service; enabled; vendor preset: enabled)
    ​​​​     Active: active (running) since Mon 2022-08-08 09:00:03 UTC; 4s ago
    ​​​​   Main PID: 1527767 (python3)
    ​​​​      Tasks: 11 (limit: 19098)
    ​​​​     Memory: 28.9M
    ​​​​     CGroup: /system.slice/thingsboard-gateway.service
    ​​​​             └─1527767 /usr/bin/python3 -c from thingsboard_gateway.tb_gateway import daemon; daemon()
    
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [mqtt_connector.py] - mqtt_connector - _on_subscribe - 326 - ()"
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [mqtt_connector.py] - mqtt_connector - _on_subscribe - 333 - "MQTT Broker Connector" subscription suc>
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [mqtt_connector.py] - mqtt_connector - _on_subscribe - 326 - ()"
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [mqtt_connector.py] - mqtt_connector - _on_subscribe - 333 - "MQTT Broker Connector" subscription suc>
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [mqtt_connector.py] - mqtt_connector - _on_subscribe - 326 - ()"
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [mqtt_connector.py] - mqtt_connector - _on_subscribe - 333 - "MQTT Broker Connector" subscription suc>
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [mqtt_connector.py] - mqtt_connector - _on_subscribe - 326 - ()"
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [mqtt_connector.py] - mqtt_connector - _on_subscribe - 333 - "MQTT Broker Connector" subscription suc>
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [tb_device_mqtt.py] - tb_device_mqtt - _on_connect - 139 - connection SUCCESS"
    ​​​​Aug 08 09:00:03 sw python3[1527767]: ""2022-08-08 09:00:03" - |INFO| - [tb_gateway_mqtt.py] - tb_gateway_mqtt - gw_subscribe_to_attribute - 175 - Subscribed to *|* with id 
    
  1. Configure the gateway
  • /etc/thingsboard-gateway/config/tb_gateway.yaml
    more
    ​​​​thingsboard:
    ​​​​  host: 192.168.1.202
    ​​​​  port: 1884
    ​​​​  remoteShell: false
    ​​​​  remoteConfiguration: false
    ​​​​  statsSendPeriodInSeconds: 3600
    ​​​​  minPackSendDelayMS: 0
    ​​​​  checkConnectorsConfigurationInSeconds: 60
    ​​​​  security:
    ​​​​    accessToken: 1NR2FQ8R3u6vIlbQc23E
    ​​​​  qos: 1
    ​​​​storage:
    ​​​​  type: memory
    ​​​​  read_records_count: 100
    ​​​​  max_records_count: 100000
    ​​​​#  type: file
    ​​​​#  data_folder_path: ./data/
    ​​​​#  max_file_count: 10
    ​​​​#  max_read_records_count: 10
    ​​​​#  max_records_per_file: 10000
    ​​​​#  type: sqlite
    ​​​​#  data_file_path: ./data/data.db
    ​​​​#  messages_ttl_check_in_hours: 1
    ​​​​#  messages_ttl_in_days: 7
    ​​​​grpc:
    ​​​​  enabled: false
    ​​​​  serverPort: 9595
    ​​​​  keepaliveTimeMs: 10000
    ​​​​  keepaliveTimeoutMs: 5000
    ​​​​  keepalivePermitWithoutCalls: true
    ​​​​  maxPingsWithoutData: 0
    ​​​​  minTimeBetweenPingsMs: 10000
    ​​​​  minPingIntervalWithoutDataMs: 5000
    ​​​​connectors:
    ​​​​  -
    ​​​​    name: MQTT Broker Connector
    ​​​​    type: mqtt
    ​​​​    configuration: mqtt.json
    
  • /etc/thingsboard-gateway/config/mqtt.json
    more
    ​​​​  {
    ​​​​  "broker": {
    ​​​​    "name": "Default Local Broker",
    ​​​​    "host": "127.0.0.1",
    ​​​​    "port": 1883,
    ​​​​    "clientId": "ThingsBoard_gateway",
    ​​​​    "maxMessageNumberPerWorker": 20,
    ​​​​    "maxNumberOfWorkers": 100,
    ​​​​    "security": {
    ​​​​     "type": "anonymous"
    ​​​​    }
    ​​​​  },
    ​​​​  "mapping": [
    ​​​​    {
    ​​​​      "topicFilter": "shellies/+/emeter/0/power",
    ​​​​      "converter": {
    ​​​​        "type": "custom",
    ​​​​        "extension": "ShellyEMMqttUplinkConverter"
    ​​​​        }
    ​​​​    },
    ​​​​    {
    ​​​​      "topicFilter": "shellies/+/emeter/0/reactive_power",
    ​​​​      "converter": {
    ​​​​        "type": "custom",
    ​​​​        "extension": "ShellyEMMqttUplinkConverter"
    ​​​​        }
    ​​​​    },
    ​​​​    {
    ​​​​      "topicFilter": "shellies/+/emeter/0/pf",
    ​​​​      "converter": {
    ​​​​        "type": "custom",
    ​​​​        "extension": "ShellyEMMqttUplinkConverter"
    ​​​​        }
    ​​​​    },
    ​​​​    {
    ​​​​      "topicFilter": "shellies/+/emeter/0/voltage",
    ​​​​      "converter": {
    ​​​​        "type": "custom",
    ​​​​        "extension": "ShellyEMMqttUplinkConverter"
    ​​​​        }
    ​​​​    },
    ​​​​    {
    ​​​​      "topicFilter": "shellies/+/emeter/0/total",
    ​​​​      "converter": {
    ​​​​        "type": "custom",
    ​​​​        "extension": "ShellyEMMqttUplinkConverter"
    ​​​​        }
    ​​​​    },
    ​​​​    {
    ​​​​      "topicFilter": "shellies/+/emeter/0/total_returned",
    ​​​​      "converter": {
    ​​​​        "type": "custom",
    ​​​​        "extension": "ShellyEMMqttUplinkConverter"
    ​​​​        }
    ​​​​    }
    ​​​​  ],
    ​​​​  "connectRequests": [
    ​​​​    {
    ​​​​      "topicFilter": "sensor/connect",
    ​​​​      "deviceNameJsonExpression": "${SerialNumber}"
    ​​​​    },
    ​​​​    {
    ​​​​      "topicFilter": "sensor/+/connect",
    ​​​​      "deviceNameTopicExpression": "(?<=sensor\/)(.*?)(?=\/connect)"
    ​​​​    }
    ​​​​  ],
    ​​​​  "disconnectRequests": [
    ​​​​    {
    ​​​​      "topicFilter": "sensor/disconnect",
    ​​​​      "deviceNameJsonExpression": "${SerialNumber}"
    ​​​​    },
    ​​​​    {
    ​​​​      "topicFilter": "sensor/+/disconnect",
    ​​​​      "deviceNameTopicExpression": "(?<=sensor\/)(.*?)(?=\/disconnect)"
    ​​​​    }
    ​​​​  ],
    ​​​​  "attributeRequests": [
    ​​​​    {
    ​​​​      "retain": false,
    ​​​​      "topicFilter": "v1/devices/me/attributes/request",
    ​​​​      "deviceNameTopicExpression": "${SerialNumber}",
    ​​​​      "attributeNameJsonExpression": "${sensorModel}"
    ​​​​    }
    ​​​​  ],
    ​​​​  "attributeUpdates": [
    ​​​​    {
    ​​​​      "retain": true,
    ​​​​      "deviceNameFilter": "SmartMeter.*",
    ​​​​      "attributeFilter": "uploadFrequency",
    ​​​​      "topicExpression": "sensor/${deviceName}/${attributeKey}",
    ​​​​      "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}"
    ​​​​    }
    ​​​​  ],
    ​​​​  "serverSideRpc": [
    ​​​​    {
    ​​​​      "deviceNameFilter": ".*",
    ​​​​      "methodFilter": "echo",
    ​​​​      "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}",
    ​​​​      "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}",
    ​​​​      "responseTimeout": 10000,
    ​​​​      "valueExpression": "${params}"
    ​​​​    },
    ​​​​    {
    ​​​​      "deviceNameFilter": ".*",
    ​​​​      "methodFilter": "no-reply",
    ​​​​      "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}",
    ​​​​      "valueExpression": "${params}"
    ​​​​    }
    ​​​​  ]
    ​​​​}
    
  • /var/lib/thingsboard_gateway/extensions/mqtt/shellyem_mqtt_uplink_converter.py
    more
    ​​​​...
    

Config and Debug

  • Config files: /etc/thingsboard-gateway/config/*
    • connected_devices.json
    • grpc_connector_1.json
    • logs.conf
    • mqtt.json
    • request.json
    • tb-cloud-chain.pem
    • rest.json
    • tb_gateway.yaml
  • Log files: /var/log/thingsboard-gateway/*
    • connector.log
    • converter.log
    • extension.log
    • service.log
    • storage.log
    • tb_connection.log

Shellies

Shelly EM

  • shellies/+/emter/0/+ (or shellies/+/emter/1/+)
    or if we configured a custom prefix in the device settings: shellies/(custom_prefix)/(device_name)/#
  • Try using software such as mqtt explorer to listen in on your mqtt broker.
    It helps you determine which messages are being sent, and the topics on which they are broadcasted.
  • Connect Shelly EM to Mosquitto Broker (allow_anonymous true)

Validations

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →