# Send data using wifi.

#### Author: Hugo Axelsson - ha223gs

This report concerns the project done for the course Applied IoT, 1DT305, at Linnaeus University. The project will describe how to set up the microcontroller LoPy4 to send data to the low code internet of things (IoT) platform Datacake using wifi.

Estimated time: **~2 hours**
Estimated price: **~€60**

## Objective

The objective for this project was:

1. Learn more about IoT and the potential of connecting the potential and limitations with a IoT devices and how to use this.
2. Create a set up repository that can be re-used for multiple projects for the future.

## Bill of material
This section lists the projects components and the corresponding price from different retailers. Note that the main reason these components was used was because of a recommended purchase and it could be that more affordable or suitable alternative exist on the market. The purchase made for this project was a package of different sensors, wires, bread board, a Pycom microcontroller with an expansion board ([see link](https://www.electrokit.com/produkt/lnu-1dt305-tillampad-iot-lopy4-and-sensors-bundle/)). To buy each individual component could increase the price but an approximated price has been included below.

1. **Microcontroller**: This report will use the LoPy 4 device from pycom to complete this tutorial. The pycom device can use LoRa, Sigfox, wifi and bluetooth which makes the controller suitable as a first controller if no clear project exists initially. The device is MicroPython enabled and will be the language that this project uses.
Price: [€38.45](https://pycom.io/product/lopy4/)

2. **Expansion Board**: This component enables a micro usb connection to the computer and ease the wiring process between the lopy and the sensors.
Price: [€17.60](https://pycom.io/product/expansion-board-3-0/)

3. **Temperature sensor**: The project used the temperature sensor MCP9700 TO-92.
Price: [€0,78](https://www.electrokit.com/produkt/mcp9700-e-to-to-92-temperaturgivare/?gclid=CjwKCAjwjJmIBhA4EiwAQdCbxmt57vSti2EiB8fYfqenZe77AscvRrO3fWPrw3p3mchShiikcUd3JRoCaoQQAvD_BwE)

4. **Breadboard**: To ease the wiring between the sensor and the microcontroller a breadboard was used.
Price: [€6.31](https://www.electrokit.com/produkt/kopplingsdack-400-anslutningar/)

5. **Cables**: Cables used for the project. Note that only five cables are needed and that you need male cables.
Price: [€3.53](https://www.electrokit.com/produkt/labbsladdar-100mm-hane-hane-30-pack/)

## Computer Setup

This chapter will describe the steps to take to set up and finish the project on Windows 10.

### Flashing Firmware
The first step is to update (flash) the firmware on the expansion board and the microcontroller.
#### Flashing the expansion board
The extension board should be updated out of the box but if this isn´t the case you can [follow this guide](https://docs.pycom.io/updatefirmware/expansionboard/) to overwrite the existing firmware of the extension board.

#### Flashing the Microcontroller

When the expansion board is flashed, mount the microcontroller to the expansion board and connect everything to the computer with a micro usb cable. To mount the microcontroller correctly onto the extension board, make sure that the "pycom" and "lopy4" logo on the microcontroller match the orientation with the "pycom" logo on the extension board (see Figure below). To ensure the functionality between the microcontroller and the computer this firmware needs to be updated. This can be done by [following this guide](https://docs.pycom.io/updatefirmware/device/)

### Chosen IDE

The IDE used for the project was Visual Studio Code (VS code) due to prior experience but an option to VS Code is Atom which also can be used. These IDE:s supports a plugin, "pyMakr", that enables the agent to upload the code to the device directly from the IDE. To enable this the terminal Node.js also need to be installed but simply [follow this step-by-step guide](https://docs.pycom.io/gettingstarted/software/vscode/) to get started with VS code. When this step is completed a first project can be performed. A recommendation is to start with [this project from pycom](https://docs.pycom.io/gettingstarted/#creating-a-project-in-pymakr). Be sure to follow the repository structure (under the heading *Creating a project in Pymakr*) that the article suggests with one possible addition of a *`keys.py`* to hide sensitive keys tied to your controller.

## The sensor configuration

To configurate the material used for the project the following circuitry was implemented.

([*source*](https://hackmd.io/S-2kPmXSRIGPSJAvmSwnfw))

The Figure below present the final configuration of each component.


Be aware that the pins you decide on using might not be compatible with the sensor output. To see what each pin on the LoPy is intended for, [see article under the section *Pinout*](https://docs.pycom.io/datasheets/development/lopy4/). For this project, pin 16 was used.

## Platform

The platform used to gather the data through wifi was the low code IoT platform Datacake. This was due to the possibility of gather data from different protocols and networks and due to the detailed documentation provided by Datacake. Datacake also enables multiple integrations from different networks (e.g helium or the things network, TTN) and for future projects this was the reason this platform was used. The course also received a redeem code to add multiple devices for free which also was a reason this platform was used (for safety reasons this can´t be shared). To send data with wifi to Datacake a the Datacake MQTT Broker was used. To get started with Datacake and MQTT [see article](https://docs.datacake.de/api/internal-mqtt/get-started) or [video presentation](https://www.youtube.com/watch?v=pUsNh1kojPQ&ab_channel=Datacake). For future project two different MQTT clients was installed, MQTT explorer and Node-red. Datacake touches both clients but with some recommendations the latter was implemented due to the browser-based flow editor. To install node red, [follow this tutorial](https://nodered.org/docs/getting-started/local) with one exception. If windows are used, copy the first command:

npm install -g --unsafe-perm node-red

in the command prompt (cmd) and not node.js or Windows Powershell. When this is installed simply run `node-red` in the cmd and then open your browser with the url:
`http://localhost:1880`
The `localgost` needs to be changed whith the adress that apperes in your terminal.

## The Code and Configuration files

Figure of the file hierarchy used for the project can be seen below.
The code used for the project is presented below and note that every constant that begins with `keys.` is personal and needs to be changed to connect to the sought wifi, MQTT address etc. From main.py file

import boot
import pycom
from machine import Pin, I2C
import time
import machine
from mqtt import MQTTClient

def connect_and_subscribe():   
    #"id","server address","port","user (token)", "password (token)", "ssl = true"
    client_id = ubinascii.hexlify(machine.unique_id())
    client = MQTTClient(client_id,"mqtt.datacake.co",port=8883,user=keys.Datacake_Token,password=keys.Datacake_Token,ssl=True)
    client.connect() # kommer kasta en exception om den inte lyckas (använd try+except för at funga upp den)
    print('Connected to MQTT broker')
    return client


def restart_and_reconnect():
    print('Failed to connect to MQTT broker. Reconnecting...')
    time.sleep(10)
    machine.reset()

def main():
    while True:        
        Package = Temperature() # The data seeked to send
        #publish to MQTT
        try:
            client.publish(Publish_Address, str(Package)) # topic, message
            print("Package sent to MQTT:"+'\t'+str(Package)+' Celsius')
            mqtt_time_point = time.time()  # reset the mqtt timer
        except OSError as e:
            restart_and_reconnect()
        time.sleep(10) # The frequency of sending data


wifi_Connection(keys.wifi_Thomas.Name,keys.wifi_Thomas.Password)
adc = machine.ADC()
apin = adc.channel(pin='P16')
pycom.heartbeat(False)
celsius = 0
try:
    client = connect_and_subscribe()
except OSError as e:
    restart_and_reconnect()
Publish_Address = Address('dtck-pub','wifi','TEMPERATURE',keys.Datacake_Publish)        

main()

The content of the boot.py file:

from network import LoRa
import time
import ubinascii
import pycom

#______________________________________________ Controller Functions ______________________________________________
def blink(Strng,Sec):
    if Strng == 'Green':
        pycom.rgbled(0x007f00) # green
    elif Strng == 'Yellow':
        pycom.rgbled(0x7f7f00) # yellow
    elif Strng == 'Red':
        pycom.rgbled(0x7f0000) # red
    elif Strng == 'White':
        pycom.rgbled(0x111111) # White
    time.sleep(Sec)
    pycom.rgbled(0x000000) # Off
#Used function to calculate the temperature in the room
def Temperature():
  millivolts = apin.voltage()
  celsius = (millivolts - 500.0) / 10.0
  return celsius
  
def wifi_Connection(Name,Password):
    from network import WLAN
    import machine
    wlan = WLAN(mode=WLAN.STA)
    wlan.connect(ssid=Name, auth=(WLAN.WPA2, Password))
    #for timeout
    #wlan.connect(ssid=Name, auth=(WLAN.WPA2, Password), timeout='time_in_ms'))
    while not wlan.isconnected():
        machine.idle()
        blink('White',1)
        print('Trying to connect to wifi',Name)
    blink("Green",2)
    print("wifi connected succesfully")
    print(wlan.ifconfig())

#______________________________________________ MQTT Functions ______________________________________________
def Address(Adrs,Device,Field,API):
    Publish_Address = Adrs +'/'+ Device +'/'+ API+ '/'+ Field
    return Publish_Address

And lastly from the lib repository and the mqtt.py file:

import usocket as socket
import ustruct as struct
from ubinascii import hexlify

class MQTTException(Exception):
    pass

class MQTTClient:

    def __init__(self, client_id, server, port=0, user=None, password=None, keepalive=0,
                 ssl=False, ssl_params={}):
        if port == 0:
            port = 8883 if ssl else 1883
        self.client_id = client_id
        self.sock = None
        self.addr = socket.getaddrinfo(server, port)[0][-1]
        self.ssl = ssl
        self.ssl_params = ssl_params
        self.pid = 0
        self.cb = None
        self.user = user
        self.pswd = password
        self.keepalive = keepalive
        self.lw_topic = None
        self.lw_msg = None
        self.lw_qos = 0
        self.lw_retain = False

    def _send_str(self, s):
        self.sock.write(struct.pack("!H", len(s)))
        self.sock.write(s)

    def _recv_len(self):
        n = 0
        sh = 0
        while 1:
            b = self.sock.read(1)[0]
            n |= (b & 0x7f) << sh
            if not b & 0x80:
                return n
            sh += 7

    def set_callback(self, f):
        self.cb = f

    def set_last_will(self, topic, msg, retain=False, qos=0):
        assert 0 <= qos <= 2
        assert topic
        self.lw_topic = topic
        self.lw_msg = msg
        self.lw_qos = qos
        self.lw_retain = retain

    def connect(self, clean_session=True):
        self.sock = socket.socket()
        self.sock.connect(self.addr)
        if self.ssl:
            import ussl
            self.sock = ussl.wrap_socket(self.sock, **self.ssl_params)
        msg = bytearray(b"\x10\0\0\x04MQTT\x04\x02\0\0")
        msg[1] = 10 + 2 + len(self.client_id)
        msg[9] = clean_session << 1
        if self.user is not None:
            msg[1] += 2 + len(self.user) + 2 + len(self.pswd)
            msg[9] |= 0xC0
        if self.keepalive:
            assert self.keepalive < 65536
            msg[10] |= self.keepalive >> 8
            msg[11] |= self.keepalive & 0x00FF
        if self.lw_topic:
            msg[1] += 2 + len(self.lw_topic) + 2 + len(self.lw_msg)
            msg[9] |= 0x4 | (self.lw_qos & 0x1) << 3 | (self.lw_qos & 0x2) << 3
            msg[9] |= self.lw_retain << 5
        self.sock.write(msg)
        #print(hex(len(msg)), hexlify(msg, ":"))
        self._send_str(self.client_id)
        if self.lw_topic:
            self._send_str(self.lw_topic)
            self._send_str(self.lw_msg)
        if self.user is not None:
            self._send_str(self.user)
            self._send_str(self.pswd)
        resp = self.sock.read(4)
        assert resp[0] == 0x20 and resp[1] == 0x02
        if resp[3] != 0:
            raise MQTTException(resp[3])
        return resp[2] & 1

    def disconnect(self):
        self.sock.write(b"\xe0\0")
        self.sock.close()

    def ping(self):
        self.sock.write(b"\xc0\0")

    def publish(self, topic, msg, retain=False, qos=0):
        pkt = bytearray(b"\x30\0\0\0")
        pkt[0] |= qos << 1 | retain
        sz = 2 + len(topic) + len(msg)
        if qos > 0:
            sz += 2
        assert sz < 2097152
        i = 1
        while sz > 0x7f:
            pkt[i] = (sz & 0x7f) | 0x80
            sz >>= 7
            i += 1
        pkt[i] = sz
        #print(hex(len(pkt)), hexlify(pkt, ":"))
        self.sock.write(pkt, i + 1)
        self._send_str(topic)
        if qos > 0:
            self.pid += 1
            pid = self.pid
            struct.pack_into("!H", pkt, 0, pid)
            self.sock.write(pkt, 2)
        self.sock.write(msg)
        if qos == 1:
            while 1:
                op = self.wait_msg()
                if op == 0x40:
                    sz = self.sock.read(1)
                    assert sz == b"\x02"
                    rcv_pid = self.sock.read(2)
                    rcv_pid = rcv_pid[0] << 8 | rcv_pid[1]
                    if pid == rcv_pid:
                        return
        elif qos == 2:
            assert 0

    def subscribe(self, topic, qos=0):
        assert self.cb is not None, "Subscribe callback is not set"
        pkt = bytearray(b"\x82\0\0\0")
        self.pid += 1
        struct.pack_into("!BH", pkt, 1, 2 + 2 + len(topic) + 1, self.pid)
        #print(hex(len(pkt)), hexlify(pkt, ":"))
        self.sock.write(pkt)
        self._send_str(topic)
        self.sock.write(qos.to_bytes(1, 'little'))
        while 1:
            op = self.wait_msg()
            if op == 0x90:
                resp = self.sock.read(4)
                #print(resp)
                assert resp[1] == pkt[2] and resp[2] == pkt[3]
                if resp[3] == 0x80:
                    raise MQTTException(resp[3])
                return

    # Wait for a single incoming MQTT message and process it.
    # Subscribed messages are delivered to a callback previously
    # set by .set_callback() method. Other (internal) MQTT
    # messages processed internally.
    def wait_msg(self):
        res = self.sock.read(1)
        self.sock.setblocking(True)
        if res is None:
            return None
        if res == b"":
            raise OSError(-1)
        if res == b"\xd0":  # PINGRESP
            sz = self.sock.read(1)[0]
            assert sz == 0
            return None
        op = res[0]
        if op & 0xf0 != 0x30:
            return op
        sz = self._recv_len()
        topic_len = self.sock.read(2)
        topic_len = (topic_len[0] << 8) | topic_len[1]
        topic = self.sock.read(topic_len)
        sz -= topic_len + 2
        if op & 6:
            pid = self.sock.read(2)
            pid = pid[0] << 8 | pid[1]
            sz -= 2
        msg = self.sock.read(sz)
        self.cb(topic, msg)
        if op & 6 == 2:
            pkt = bytearray(b"\x40\x02\0\0")
            struct.pack_into("!H", pkt, 2, pid)
            self.sock.write(pkt)
        elif op & 6 == 4:
            assert 0

    # Checks whether a pending message from server is available.
    # If not, returns immediately with None. Otherwise, does
    # the same processing as wait_msg.
    def check_msg(self):
        self.sock.setblocking(False)
        return self.wait_msg()

## Transmitting the data

The data is currently being send with the wifi protocol and is updated each 15 seconds. To achieve this, the used protocol was MQTT. The high transfer frequency was due to that the device is not active for long periods of one day and a high send cap at 500 datapoints per day for the free version on Datacake ([*source*](https://datacake.co/pricing)). This frequency could be lowered to 1 datapoint per minute due to the limit on Datacakes graph builder at 1 datapoint per minute.

## Presenting the data

The figure below present how the dashboard on Datacake is set up. This includes a graph with the temperature on the vertical axis and the time on the horizontal axis. The resolution in the graph is one minute and the latest temperature is presented to the right of the graph.

## Finalizing the design

The goal of this course was to learn about IoT and be able to get a basic understanding of how it can be implemented on a smaller scale as home projects. To use this course to get a good starting ground for my own future projects. Unfortunately the difficulty of the project was not at the level I initially visualized it to be. This was in part of a slow shipment from China, but the primary reason was the summer vacation. Some improvment I would have liked to implement would have been more sensors and a deeper understanding on how to send data with different formats (e.g Json and not like a string) and how every step is interconnected. Due to sometime limitation this report didn´t include the LoRa protocol implemented, and this is also a setback.

All in all, the progress made and the knowledge gathered throughout this course was a great step in my future projects and have awaken a drive to keep learning about computer structures and how it all interconnects.

Due to the simplicity of the project the result has been presented above with Figures of the assembled device and the integration in the datacake platform but below are some reminders.