--- title: Installation & Configuration of IoT Gateway tags: thingsboard-gateway image: --- # Getting started with ThingsBoard IoT Gateway :::info **What is Thingsboard IoT Gateway?** * Devices > --- modbus --- > converter / connector > * ... ::: [toc] ## Installation ### Docker + Linux > Prerequisites: docker ce + docker-compose > Docker basics 1. How to Do a Clean Restart of a Docker Instance [details](https://docs.tibco.com/pub/mash-local/4.3.0/doc/html/docker/GUID-BD850566-5B79-4915-987E-430FC38DAAE4.html) * Stop the container(s) using the following command: ``docker-compose down`` * Delete all containers using the following command: ``docker rm -f $(docker ps -a -q) `` * Delete all volumes using the following command: ``docker volume rm $(docker volume ls -q)`` ==Note: Deleting volumes will wipe out their data. Back up any data that you need before deleting a container.== * Restart the containers using the following command: ``docker-compose up -d `` 2. ... > docker-compose.yml (template [here](https://github.com/thingsboard/thingsboard-gateway/blob/master/docker/docker-compose.yml)) ```yaml! version: '3.0' services: gw: restart: always image: "thingsboard/tb-gateway" # Ports (Required for REST connector) ports: - "5000:5000" volumes: - ~/.tb-gateway/config:/thingsboard_gateway/config - ~/.tb-gateway/logs:/thingsboard_gateway/logs - ~/.tb-gateway/extensions:/thingsboard_gateway/extensions devices: - "/dev/ttyS0:/dev/ttyS0" ``` 1. working directory with the file "docker-compose.yml": ``cd ~/workspace/tb`` 2. ``docker-compose up -d`` 3. ``docker-compose logs --tail 10`` 4. ``star@gateway:~/workspace/tb$ docker exec -it tb-gw-1 bash`` ``` root@06154190eb83:/# pip3 install redis Collecting redis Downloading redis-4.4.2-py3-none-any.whl (237 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 237.8/237.8 kB 1.5 MB/s eta 0:00:00 Collecting async-timeout>=4.0.2 Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB) Installing collected packages: async-timeout, redis Successfully installed async-timeout-4.0.2 redis-4.4.2 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip available: 22.3 -> 23.0 [notice] To update, run: python3 -m pip install --upgrade pip root@06154190eb83:/# pip3 install sqlalchemy Collecting sqlalchemy Downloading SQLAlchemy-2.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.7/2.7 MB 3.1 MB/s eta 0:00:00 Collecting greenlet!=0.4.17 Downloading greenlet-2.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 613.7/613.7 kB 4.0 MB/s eta 0:00:00 Requirement already satisfied: typing-extensions>=4.2.0 in /root/.local/lib/python3.10/site-packages (from sqlalchemy) (4.4.0) Installing collected packages: greenlet, sqlalchemy Successfully installed greenlet-2.0.2 sqlalchemy-2.0.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip available: 22.3 -> 23.0 [notice] To update, run: python3 -m pip install --upgrade pip ``` > console error ``` tb-gw-1 | ""2023-01-31 04:50:23" - |ERROR| - [modbus_connector.py] - modbus_connector - __process_slaves - 342 - int() argument must be a string, a bytes-like object or a real number, not 'NoneType'" tb-gw-1 | Traceback (most recent call last): tb-gw-1 | File "/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 311, in __process_slaves tb-gw-1 | self.__connect_to_current_master(device) tb-gw-1 | File "/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 357, in __connect_to_current_master tb-gw-1 | device.config['master'], device.config['available_functions'] = self.__configure_master(device.config) tb-gw-1 | File "/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 419, in __configure_master tb-gw-1 | master = ModbusSerialClient(method=current_config["method"], tb-gw-1 | File "/root/.local/lib/python3.10/site-packages/pymodbus/client/serial.py", line 196, in __init__ tb-gw-1 | super().__init__(framer=framer, **kwargs) tb-gw-1 | File "/root/.local/lib/python3.10/site-packages/pymodbus/client/base.py", line 120, in __init__ tb-gw-1 | self.params.retries = int(retries) tb-gw-1 | TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' tb-gw-1 | ""2023-01-31 04:50:28" - |ERROR| - [modbus_connector.py] - modbus_connector - __process_slaves - 342 - int() argument must be a string, a bytes-like object or a real number, not 'NoneType'" tb-gw-1 | Traceback (most recent call last): tb-gw-1 | File "/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 311, in __process_slaves tb-gw-1 | self.__connect_to_current_master(device) tb-gw-1 | File "/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 357, in __connect_to_current_master tb-gw-1 | device.config['master'], device.config['available_functions'] = self.__configure_master(device.config) tb-gw-1 | File "/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 419, in __configure_master tb-gw-1 | master = ModbusSerialClient(method=current_config["method"], tb-gw-1 | File "/root/.local/lib/python3.10/site-packages/pymodbus/client/serial.py", line 196, in __init__ tb-gw-1 | super().__init__(framer=framer, **kwargs) tb-gw-1 | File "/root/.local/lib/python3.10/site-packages/pymodbus/client/base.py", line 120, in __init__ tb-gw-1 | self.params.retries = int(retries) tb-gw-1 | TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' tb-gw-1 | ""2023-01-31 04:50:33" - |ERROR| - [modbus_connector.py] - modbus_connector - __process_slaves - 342 - int() argument must be a string, a bytes-like object or a real number, not 'NoneType'" tb-gw-1 | Traceback (most recent call last): tb-gw-1 | File "/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 311, in __process_slaves tb-gw-1 | self.__connect_to_current_master(device) tb-gw-1 | File "/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 357, in __connect_to_current_master tb-gw-1 | device.config['master'], device.config['available_functions'] = self.__configure_master(device.config) tb-gw-1 | File "/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 419, in __configure_master tb-gw-1 | master = ModbusSerialClient(method=current_config["method"], tb-gw-1 | File "/root/.local/lib/python3.10/site-packages/pymodbus/client/serial.py", line 196, in __init__ tb-gw-1 | super().__init__(framer=framer, **kwargs) tb-gw-1 | File "/root/.local/lib/python3.10/site-packages/pymodbus/client/base.py", line 120, in __init__ tb-gw-1 | self.params.retries = int(retries) tb-gw-1 | TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' ``` ``` root@06154190eb83:/# pip3 install protobuf --upgrade Requirement already satisfied: protobuf in /usr/local/lib/python3.10/site-packages (3.20.0) Collecting protobuf Downloading protobuf-4.21.12-cp37-abi3-manylinux2014_x86_64.whl (409 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 409.8/409.8 kB 2.1 MB/s eta 0:00:00 Installing collected packages: protobuf Attempting uninstall: protobuf Found existing installation: protobuf 3.20.0 Uninstalling protobuf-3.20.0: Successfully uninstalled protobuf-3.20.0 Successfully installed protobuf-4.21.12 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip available: 22.3 -> 23.0 [notice] To update, run: python3 -m pip install --upgrade pip ``` > try to make freeze and see that versions of libs ``` root@e65a1a2bd181:/# pip3 freeze ``` ### Ubuntu 20 / Debian 11 * (optional but recommended) ``sudo apt update`` #### 1. Download the deb file (/home/swae/workspace/tb-gw) ```! wget https://github.com/thingsboard/thingsboard-gateway/releases/latest/download/python3-thingsboard-gateway.deb ``` #### 2. Install the gateway using apt ``` sudo apt install ./python3-thingsboard-gateway.deb -y ``` #### 3. Check gateway status ``` sudo systemctl status thingsboard-gateway ``` :::spoiler details ``` star@debian-vm:~/workspace/tb-gw$ sudo systemctl status thingsboard-gateway.service ● thingsboard-gateway.service - ThingsBoard Gateway Loaded: loaded (/etc/systemd/system/thingsboard-gateway.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2023-01-17 12:02:18 CST; 20min ago Main PID: 20949 (python3) Tasks: 10 (limit: 2341) Memory: 30.7M CPU: 5.435s CGroup: /system.slice/thingsboard-gateway.service └─20949 /usr/bin/python3 -c from thingsboard_gateway.tb_gateway import daemon; daemon() Jan 17 12:02:19 debian-vm python3[20949]: File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/paho/mqtt/re> Jan 17 12:02:19 debian-vm python3[20949]: assert identifier in self.names.keys(), identifier Jan 17 12:02:19 debian-vm python3[20949]: AssertionError: 5 Jan 17 12:02:19 debian-vm python3[20949]: ""2023-01-17 12:02:19" - |ERROR| - [tb_device_mqtt.py] - tb_device_mqtt - _on_connec> Jan 17 12:02:19 debian-vm python3[20949]: ""2023-01-17 12:02:19" - |INFO| - [tb_updater.py] - tb_updater - check_for_new_versi> Jan 17 12:02:19 debian-vm python3[20949]: ""2023-01-17 12:02:19" - |INFO| - [tb_updater.py] - tb_updater - check_for_new_versi> Jan 17 12:02:19 debian-vm python3[20949]: [===UPDATE===] Jan 17 12:02:19 debian-vm python3[20949]: New version v3.2 is available! Jan 17 12:02:19 debian-vm python3[20949]: [===UPDATE===] Jan 17 12:02:19 debian-vm python3[20949]: " ``` ::: #### (optional) 4. Hot Reloader > Enable hot reloader to restart Gateway every time when we edit any project file when using ++Gateway for development++. ``python3 ./thingsboard_gateway/tb_gateway.py true`` --- source code ``python3 /usr/lib/python3/dist-packages/thingsboard_gateway/tb_gateway.py true`` ## Configuration ### Directory structure * * path: /etc/thingsboard-gateway/config/* * **tb_gateway.yaml** * modbus_serial.json #### Disable Statistics * tb- ``` ``` * * statistics.json --- /usr/lib/python3/dist-packages/thingsboard_gateway/config/statistics.json * * ... ## Custom ### iAeris20 setting gateway 讀取 iAeris register 的資料經由hex轉換成int時會以4位數呈現小數點後兩位，此呈現方式在顯示上並不符合閱讀邏輯。 暫時解決方案為在 bytes_modbus_uplink_converter.py 檔案中添加下方程式碼強制轉換，後續須考慮新方法以解決不同機型產生的數字所代表的含意。 ``` star@gateway:~/workspace/tb$ docker exec -it tb-gw-1 bash root@abfd2e6c9960:/# cd /thingsboard_gateway/connectors/modbus/ ``` Edit bytes_modbus_uplink_converter.py (+line 83) ```python=83 decoded_data=float(decoded_date)/100 ``` p.s. docker在更新過後需重啟docker才能生效 Result: before: `` tb-gw-1 | ""2023-02-21 03:30:37" - |INFO| - [modbus_connector.py] - modbus_connector - run - 651 - {'deviceName': 'iAeris 20', 'deviceType': 'default', 'telemetry': [{'temperature': 2660}, {'humidity': 3590}], 'attributes': []}" `` after: `` tb-gw-1 | ""2023-02-21 06:41:00" - |INFO| - [modbus_connector.py] - modbus_connector - run - 651 - {'deviceName': 'iAeris 20', 'deviceType': 'default', 'telemetry': [{'temperature': 26.9}, {'humidity': 34.9}], 'attributes': []}" `` ## Troubleshooting > ``sudo stty -a -F /dev/ttyS0`` ``` speed 19200 baud; rows 0; columns 0; line = 0; intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; discard = ^O; min = 0; time = 0; -parenb -parodd -cmspar cs8 -hupcl -cstopb cread clocal -crtscts -ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl -ixon -ixoff -iuclc -ixany -imaxbel -iutf8 -opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0 -isig -icanon -iexten -echo -echoe -echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke -flusho -extproc ``` ``` connectors: # - # name: MQTT Broker Connector # type: mqtt # configuration: mqtt.json # - # name: Modbus Connector # type: modbus # configuration: modbus.json # - name: Modbus Connector type: modbus configuration: modbus_serial.json ``` ``` Jan 17 12:49:25 debian-vm python3[21319]: from pymodbus.constants import Defaults Jan 17 12:49:25 debian-vm python3[21319]: ModuleNotFoundError: No module named 'pymodbus' Jan 17 12:49:25 debian-vm python3[21319]: Modbus library not found - installing... Jan 17 12:49:25 debian-vm python3[21319]: ""2023-01-17 12:49:25" - |ERROR| - [tb_device_mqtt.py] - tb_device_mqtt - _on_connect - 223 - connection FAIL with> Jan 17 12:49:25 debian-vm python3[21319]: ""2023-01-17 12:49:25" - |INFO| - [tb_updater.py] - tb_updater - check_for_new_version - 84 - v3.2" Jan 17 12:49:25 debian-vm python3[21319]: ""2023-01-17 12:49:25" - |INFO| - [tb_updater.py] - tb_updater - check_for_new_version - 86 - ``` ``` Jan 17 12:49:25 debian-vm python3[21319]: [===UPDATE===] Jan 17 12:49:25 debian-vm python3[21319]: New version v3.2 is available! Jan 17 12:49:25 debian-vm python3[21319]: [===UPDATE===] Jan 17 12:49:25 debian-vm python3[21319]: " Jan 17 12:50:27 debian-vm python3[21326]: Collecting pymodbus>=3.0.0 Jan 17 12:50:42 debian-vm python3[21326]: Downloading pymodbus-3.1.0-py2.py3-none-any.whl (163 kB) Jan 17 12:50:43 debian-vm python3[21326]: Installing collected packages: pymodbus Jan 17 12:50:43 debian-vm python3[21326]: WARNING: The scripts pymodbus.console, pymodbus.server and pymodbus.simulator are installed in '/var/lib/thingsb> Jan 17 12:50:43 debian-vm python3[21326]: Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. Jan 17 12:50:43 debian-vm python3[21326]: Successfully installed pymodbus-3.1.0 ``` * error log: tail -f /var/log/thingsboard-gateway/connector.log ``` Jan 17 13:02:10 debian-vm python3[21319]: self.__connect_to_current_master(device) Jan 17 13:02:10 debian-vm python3[21319]: File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 357, in __> Jan 17 13:02:10 debian-vm python3[21319]: device.config['master'], device.config['available_functions'] = self.__configure_master(device.config) Jan 17 13:02:10 debian-vm python3[21319]: File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/modbus/modbus_connector.py", line 419, in __> Jan 17 13:02:10 debian-vm python3[21319]: master = ModbusSerialClient(method=current_config["method"], Jan 17 13:02:10 debian-vm python3[21319]: File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/serial.py", line 193, in _> Jan 17 13:02:10 debian-vm python3[21319]: super().__init__(framer=framer, **kwargs) Jan 17 13:02:10 debian-vm python3[21319]: File "/var/lib/thingsboard_gateway/.local/lib/python3.9/site-packages/pymodbus/client/base.py", line 120, in __i> Jan 17 13:02:10 debian-vm python3[21319]: self.params.retries = int(retries) Jan 17 13:02:10 debian-vm python3[21319]: TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType' ``` ``sudo pip3 install pymodbus pyserial twisted --upgrade`` ``` Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.9/dist-packages (from twisted) (22.2.0) Requirement already satisfied: six in /usr/lib/python3/dist-packages (from Automat>=0.8.0->twisted) (1.16.0) Requirement already satisfied: idna>=2.5 in /usr/lib/python3/dist-packages (from hyperlink>=17.1.1->twisted) (2.10) Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from zope.interface>=4.4.2->twisted) (52.0.0) Installing collected packages: zope.interface, typing-extensions, incremental, hyperlink, constantly, Automat, twisted, pyserial, pymodbus Successfully installed Automat-22.10.0 constantly-15.1.0 hyperlink-21.0.0 incremental-22.10.0 pymodbus-3.1.0 pyserial-3.5 twisted-22.10.0 typing-extensions-4.4.0 zope.interface-5.5.2 ``` ``` root@gateway:~# python /testmode/_1_Python3_Codes/_5_PySerial-Receive_String.py send：b'\xfe\x03\x00\x1a\x00\x02\xf1\xc3' get：[b'\xfe\x03\x04\t\x06\x16&\x99\x1b'] - id 0xfe 254 any sensor ``` > modpoll ``` ./modpoll -b 19200 -p none -m rtu -a 1 -r 0x1b -c 1 /dev/ttyS0 modpoll 3.10 - FieldTalk(tm) Modbus(R) Master Simulator Copyright (c) 2002-2021 proconX Pty Ltd Visit https://www.modbusdriver.com for Modbus libraries and tools. Protocol configuration: Modbus RTU, FC3 Slave configuration...: address = 1, start reference = 27, count = 1 Communication.........: /dev/ttyS0, 19200, 8, 1, none, t/o 1.00 s, poll rate 1000 ms Data type.............: 16-bit register, output (holding) register table -- Polling slave... (Ctrl-C to stop) [27]: 2320 -- Polling slave... (Ctrl-C to stop) [27]: 2320 -- Polling slave... (Ctrl-C to stop) [27]: 2320 ``` > linux-serial-test (helpless wiht RS-485 mode :<) ``` star@gateway:/testmode$ ./linux-serial-test -q -s -e -p /dev/ttyS0 Linux serial test app Error setting RS-485 mode: Inappropriate ioctl for device /dev/ttyS0: No data received for 2.2s. /dev/ttyS0: No data received for 3.3s. /dev/ttyS0: No data received for 4.3s. /dev/ttyS0: No data received for 5.4s. /dev/ttyS0: No data received for 6.5s. ``` ### Log files * /etc/thingsboard-gateway/config/logs.conf * /var/log/thingsboard-gateway/* ``` sudo journalctl --follow -u thingsboard-gateway.service sudo journalctl -fu thingsboard-gateway.service ``` ## References * [What is ThingsBoard IoT Gateway?](https://thingsboard.io/docs/iot-gateway/what-is-iot-gateway/) * [Getting Started with ThingsBoard IoT Gateway](https://thingsboard.io/docs/iot-gateway/getting-started/) * [tb-gateway config for modbus](https://thingsboard.io/docs/iot-gateway/config/modbus/) --- ==!!!重要!!!== * [mbpoll user manual](https://www.modbustools.com/mbpoll-user-manual.html#_modbus_poll) * https://github.com/thingsboard/thingsboard-gateway/issues/1041 ## Star's Lab (手上沒有硬體的做法) * Windows Modbus slave simulator * Linux Modbus master simulator [here](https://www.modbusdriver.com/modpoll.html)