# LAB 3: sending sensors data with LoRaWAN via TTN to Ubidots
> KIC, June 2019
> version 11-06-2019 19:36

This lab aims to offer you an hands-on experience with the complete process of generating data from an IoT device, send the information through a LoRaWAN gateway to the TTN server, and how to collect and visualize the data.

The overall map of what we will be using is shown in the figure below.

# Step 1: setting up the TTN (The Things Network) server

As describe before in class, in TTN there are two basic elements: the *applications* and the *devices*.

==We already set-up an application and associated to it the devices you will use in this lab. If you want to see an overview of how to do it, [click here](https://hackmd.io/QGdgS_bVT4uaiT-L7MJUfg)==

To use TTN, you need an account.
1. [Create an account](https://account.thethingsnetwork.org/register).
2. Select [Console](https://console.thethingsnetwork.org/) from the top menu.
3. Select [Applications](https://console.thethingsnetwork.org/applications). You will have not application at the moment.

==Give your username to us so that we can add you to the application we created.==

Eventually, you will see something like this:

# Step 2: feeding the TTN server with data from the sensors

In this step we will use the LoPy plus the PySense board to periodically send the temperature, and luminosity (lux).
The LoPy are already registered and are running the code available [here](https://github.com/pmanzoni/KIC2019/tree/master/Lab_code).

**When you power up your device it will automatically start generating data to TTN.**

Now, go in the "Data" section of your TTN Application. You will see something like:

If you click on any of the lines of the data, you'll get:


where you can find a lot of information regarding the sending of you LoRa message.

If you check the "Payload" field, you will see a sequence of bytes; these bytes are interpreted in the "Fields" field. This translation is perfomed from TTN using the code in section **Payload Formats**. Click on that link to see the code:


(It is also available [here](https://github.com/pmanzoni/KIC2019/tree/master/decode))

**Now, TTN does not store the incoming data for a long time. If we want to keep these data, process and visualize them, we need to get them and store them somewhere.**

TTN can be accessed using MQTT. We will first of all (**Step 3**) see how to acces the data using MQTT.

Finally, (**Step 4**) we will collect data from TTN and send it directly to Ubidots.

------

# Step 3: accessing data from TTN
TTN does not store the incoming data for a long time. If we want to keep these data, process and visualize them, we need to get them and store them somewhere.

TTN can be accessed using MQTT. We will first of all write the code necessary to access TTN through MQTT and read the incoming data.

> All the details of the TTN MQTT API, can be found here: https://www.thethingsnetwork.org/docs/applications/mqtt/quick-start.html

## Simple data collection using MQTT
In this Lab you will work with MQTT using the [**MQTT Paho** library](https://www.eclipse.org/paho/clients/python/) for Python. Programming will be done using a cloud development environment called [**repl.it**](https://repl.it/login). If you don't already have an account for it, you have to create one right now (it takes 3 seconds). repl.it will take care of installing the MQTT Paho library.

Once in your `repl.it` account, click on the  button and then select Python. You will see something like this:

The box on the right will show the execution of the code.
**IMPORTANT:**
* The code has to be inserted in the box indicated with the blu line in the Figure.
* When executing the code of the other examples you will have to create a "new repl"; **do not** add files in the box indicated with the red line in the Figure.

The code to be used is the one below; it is also available in the github repository named as [step3.py](https://github.com/pmanzoni/KIC2019/blob/master/step3.py).

==Remember to first properly set the values for the username (`TTN_USERNAME`) which is the **Application ID** and the password (`TTN_PASSWORD`) which is the Application **Access Key**, in the bottom part of the _Overview_ section of the "Application" window.==

import sys
import time
import base64

import json
import struct

import paho.mqtt.client as mqtt

THE_BROKER = "asia-se.thethings.network"
THE_TOPIC = "+/devices/+/up"

# SET HERE THE VALUES OF YOUR APP AND DEVICE:
# TTN_USERNAME is the Application ID
TTN_USERNAME = "VOID"
# TTN_PASSWORD is the Application Access Key, in the bottom part of the Overview section of the “Application” window.
TTN_PASSWORD = "VOID"

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected to ", client._host, "port: ", client._port)
    print("Flags: ", flags, "return code: ", rc)

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe(THE_TOPIC)

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):

    themsg = json.loads(msg.payload.decode("utf-8"))

    payload_raw = themsg["payload_raw"]
    payload_plain = base64.b64decode(payload_raw)
    vals = struct.unpack(">ff", payload_plain)

    dev_id = themsg["dev_id"]
    gtw_id = themsg["metadata"]["gateways"][0]["gtw_id"]
    rssi = themsg["metadata"]["gateways"][0]["rssi"]

    print("%s, gtw=%s, rssi=%d" % (dev_id, gtw_id, rssi))
    print("@%s >> temp=%.3f lux=%.3f" % (time.strftime("%H:%M:%S"), vals[0], vals[1]))




client = mqtt.Client()

# Let's see if you inserted the required data
if TTN_USERNAME == 'VOID':
    print("You must set the values of your app and device first!!")
    sys.exit()
client.username_pw_set(TTN_USERNAME, password=TTN_PASSWORD)

client.on_connect = on_connect
client.on_message = on_message

client.connect(THE_BROKER, 1883, 60)

client.loop_forever()

Now, with this code executing, **and your device generating data to TTN (as before)** you should start seeing data coming to your `repl` console:


# Step 4: Data collection using Ubidots
> https://help.ubidots.com/developer-guides/integrate-your-ttn-data-with-ubidots-simple-setup

In this step will will connect TTN with Ubidots, a web plaftorm used to store and visualize data. For these experiments we will use the [Ubidots](https://ubidots.com/) plaftorm.

## Creating a Ubidots account
You will have to first create your free account in the Ubidots platform here: https://app.ubidots.com/accounts/signup/

## Adding the Ubidots Integration from TTN
In TTN, in the the Application section of the web, click on "Integrations", then click on "Add integration" and select "Ubidots."

1) 

2) 

3) 

Next, give a customized name to your new integration (in this guide we simply called it "ubi-integration").

Then, select "default key" in the Access Key dropdown menu. The default key represents a "password" that is used to authenticate your application in TTN.

Finally, enter your Ubidots TOKEN where indicated in the TTN user interface and click on the button "Add Integration"


Where to get this TOKEN? Go to your web interface in Ubidots and click on the icon of your user in the top right corner of the window


a menu will appear where you have to select the "API Credentials" item and then the Tokens window will appear

## Visualize your data in Ubidots
Now, you will be able to see your LoRaWAN devices automatically created in your Ubidots account. Please note this integration will automatically use your DevEUI as the "Device API Label," which is the unique identifier within Ubidots used to automatically create and identify different devices.

Because Ubidots automatically assigns the Device Name equal to the Device API Label, you will see that the device does not have a human-readable name.


Feel free to change it to your liking.

Click now on the device, you well see something like this:


You can choose any of the variable by click on any of them to see the evolution of the values. For example:

Clicking on the  button you can choose other types of plots. Try them.

## Visualize your data in Ubidots using the Dashboard

Select the Ubidots dashboard by clicking on the top bar:


you will get to a section where you can add widgets to see your data in real-time.

**For example, try to re-create a dashboard like the one below:**