# LoRaWAN Lab
###### tags: `TA Stuff 2021` `Pycom` `ESP32` `ttn` `TIG-stack`

---

Credit goes to **Pietro Manzoni** https://hackmd.io/@pmanzoni for the great tutorial.

---

This Lab session will guide you through working with **The Things Networks** to (step 1) send sensor data over LoRaWAN to a cloud server and how to (step 2) process and visualize those data.

**All the code necessary for this Lab session is available at [https://github.com/pmanzoni/hackmd_code](https://github.com/pmanzoni/hackmd_code)** in folder `code`.

# Step 1: sending sensor data over LoRaWAN to a cloud server

## "The Things Network" cloud server

The Things Network is a web service that enables low power Devices to use long range Gateways to connect to an open-source, decentralized Network to exchange data with Applications.

You will manage your applications and devices via [The Things Network Console](https://console.thethingsnetwork.org/).

### Create an Account

To use the console, you need an account.
1. [Create an account](https://account.thethingsnetwork.org/register).
2. Select [Console](https://console.thethingsnetwork.org/) from the top menu.

### Add an Application in the Console

Add your first The Things Network Application.


1. In the [Console](https://console.thethingsnetwork.org/), click [add application](https://console.thethingsnetwork.org/applications/add)

* For **Application ID**, choose a unique ID of lower case, alphanumeric characters and nonconsecutive `-` and `_` (e.g., `hi-world`).
* For **Description**, enter anything you like (e.g. `Hi, World!`).


2. Click **Add application** to finish.

You will be redirected to the newly added application, where you can find the generated **Application EUI** and default **Access Key** which we'll need later.


> If the Application ID is already taken, you will end up at the Applications overview with the following error. Simply go back and try another ID.


### Register the Device

The Things Network supports the two LoRaWAN mechanisms to register devices: Over The Air Activation (OTAA) and Activation By Personalization (ABP). In this lab, we will use **OTAA**. This is more reliable because the activation will be confirmed and more secure because the session keys will be negotiated with every activation. *(ABP is useful for workshops because you don't have to wait for a downlink window to become available to confirm the activation.)*

1. On the Application screen, scroll down to the **Devices** box and click on **register device**.


* As **Device ID**, choose a unique ID (for this application) of lower case, alphanumeric characters and nonconsecutive `-` and `_` (e.g., `my-device1`).
* As **Device EUI**, you have to use the value you get by executing in your LoPy the code `getdeveui.py`


2. Click **Register**.

You will be redirected to the newly registered device.

3. On the device screen, select **Settings** from the top right menu.


* You can give your device a description like `My first TTN device`
* Check that *Activation method* is set to *OTAA*.
* Uncheck **Frame counter checks** at the bottom of the page.

> **Note:** This allows you to restart your device for development purposes without the routing services keeping track of the frame counter. This does make your application vulnerable for replay attacks, e.g. sending messages with a frame counter equal or lower than the latest received. Please do not disable it in production.

4. Click **Save** to finish.

You will be redirected to the device, where you can find the **Device Address**, **Network Session Key** and **App Session Key** that we'll need next.


## Sending data to TTN with the LoPy

In this step we will use the device (the LoPy plus the PySense) registered in the step before to periodically send the sensed temperature, humidity and luminosity (lux).

...

# SET HERE THE VALUES OF YOUR APP AND DEVICE
THE_APP_EUI = '70B3D57ED0024BEE'
THE_APP_KEY = '6DDA267B9F9A51C570A5804A2F51B905'


def join_lora(force_join = False):
    '''Joining The Things Network '''
    print('Joining TTN')

    # restore previous state
    if not force_join:
        lora.nvram_restore()

    if not lora.has_joined() or force_join == True:

        # create an OTA authentication params
        app_eui = binascii.unhexlify(THE_APP_EUI.replace(' ','')) # these settings can be found from TTN
        app_key = binascii.unhexlify(THE_APP_KEY.replace(' ','')) # these settings can be found from TTN

        # join a network using OTAA if not previously done
        lora.join(activation=LoRa.OTAA, auth=(app_eui, app_key), timeout=0)

        # wait until the module has joined the network
        while not lora.has_joined():
            time.sleep(2.5)

        # saving the state
        lora.nvram_save()

        # returning whether the join was successful
        if lora.has_joined():
            flash_led_to(GREEN)
            print('LoRa Joined')
            return True
        else:
            flash_led_to(RED)
            print('LoRa Not Joined')
            return False

    else:
        return True

pycom.heartbeat(False) # Disable the heartbeat LED

# Getting the LoRa MAC
lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868)
print("Device LoRa MAC:", binascii.hexlify(lora.mac()))

flash_led_to(YELLOW)
# joining TTN
join_lora(True)

py = Pysense()
tempHum = SI7006A20(py)
ambientLight = LTR329ALS01(py) 

while True:
    # create a LoRa socket
    s = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
    s.setsockopt(socket.SOL_LORA, socket.SO_DR, 0)
    s.setblocking(True)

    temperature = tempHum.temperature()
    humidity = tempHum.humidity()
    luxval = raw2Lux(ambientLight.light())

    print("Read sensors: temp. {} hum. {} lux: {}".format(temperature, humidity, luxval))

    # Packing sensor data as byte sequence using 'struct'
    # Data is represented as 3 float values, each of 4 bytes, byte orde 'big-endian'
    # for more infos: https://docs.python.org/3.6/library/struct.html
    payload = struct.pack(">fff", temperature, humidity, luxval)

    s.send(payload)
    flash_led_to(GREEN)

    time.sleep(15)

Running the above code will produce something like:

Device LoRa MAC: b'70b3d.....a6c64'
Joining TTN
LoRa Joined
Read sensors: temp. 30.14548 hum. 57.33438 lux: 64.64554
Read sensors: temp. 30.1562 hum. 57.31149 lux: 64.64554
...

==Joining the TTN may take a few seconds==

Now, go in the "Data" section of your TTN Application. You will see:
The first line in the bottom is the message that represents the conection establishment and the other lines the incoming data.

If you click on any of the lines of the data, you'll get:


where you can find a lot of information regarding the sending of you LoRa message.

We are basically sending every 15 seconds the values of temperature, humidity and luminosity (lux) "compressed" as a sequence of 4*3= 12 bytes

payload = struct.pack(">fff", temperature, humidity, luxval)

Now, to allow TTN to interpret these sequence of bytes we have to go the the section **Payload Format** and insert the necessary code:


Go back to the Data window in TTN and start again you LoPy.

You will see that now even lines show some more infos:


and if you click on any of the lines you will see:

that is, the data in readable format.

## Data collection using MQTT (in Python)

:::danger
TTN does not store the incoming data for a long time. If we want to keep these data, process and visualize them, we need to get them and store them somewhere.
:::

TTN can be accessed using MQTT. All the details of the TTN MQTT API, can be found here: https://www.thethingsnetwork.org/docs/applications/mqtt/quick-start.html

Using **python** (not micropython) you will access TTN through MQTT and read the incoming data.

...

import paho.mqtt.client as mqtt

THE_BROKER = "eu.thethings.network"
THE_TOPIC = "+/devices/+/up"

# SET HERE THE VALUES OF YOUR APP AND DEVICE:
# TTN_USERNAME is the Application ID
TTN_USERNAME = "VOID"
# TTN_PASSWORD is the Application Access Key, in the bottom part of the Overview section of the  "Application"  window.
TTN_PASSWORD = "VOID"

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected to ", client._host, "port: ", client._port)
    print("Flags: ", flags, "return code: ", rc)

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe(THE_TOPIC)

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):

    themsg = json.loads(msg.payload.decode("utf-8"))

    payload_raw = themsg["payload_raw"]
    payload_plain = base64.b64decode(payload_raw)
    vals = struct.unpack(">fff", payload_plain)

    gtw_id = themsg["metadata"]["gateways"][0]["gtw_id"]
    rssi = themsg["metadata"]["gateways"][0]["rssi"]

    print("%s, rssi=%d" % (gtw_id, rssi))
    print("@%s >> temp=%.3f hum=%.3f lux=%.3f" % (time.strftime("%H:%M:%S"), vals[0], vals[1], vals[2]))


client = mqtt.Client()

client.username_pw_set(TTN_USERNAME, password=TTN_PASSWORD)

client.on_connect = on_connect
client.on_message = on_message

client.connect(THE_BROKER, 1883, 60)

client.loop_forever()

Now, with this code executing, **and your device generating data to TTN (as before)** you should start seeing data coming to you:


## Data collection using Ubidots
<!--
https://help.ubidots.com/en/articles/2362758-integrate-your-ttn-data-with-ubidots-simple-setup
-->

Now you're just 1 step away from seeing your data in Ubidots.

==Open a web page with the Ubidots account you created in the previous sessions.==

Within your TTN account, with the decoded active, click on "Integrations":

then click on "Add integration" and select "Ubidots."


Next, give a customized name to your new integration (for example "ubi-integration").

Then, select "default key" in the Access Key dropdown menu. The default key represents a "password" that is used to authenticate your application in TTN.

Finally, enter your Ubidots TOKEN

where indicated in the TTN user interface.


You'll obtain something like:

### Visualize your data in Ubidots

Finally, upon successful creation of the decoder for your application's data payload with the TTN integration, you will be able to see your LoRaWAN devices automatically created in your Ubidots account.

Please note this integration will automatically use your DevEUI as the "API Label," which is the unique identifier within Ubidots used to automatically create and identify different devices:

Because Ubidots automatically assigns the Device Name equal to the Device API Label, you will see that the device does not have a human-readable name. Feel free to change it to your liking.

----
----