# IoT Lab 1: MQTT with Python
###### tags: `SRM` `Labs`
> curso 2019/2020

This lab aims to offer you an hands-on experience with MQTT. You will perform experiments that will allow you to learn how to "publish" data and "subscribe" to get data.

**All the code necessary for this Lab session is available at [bit.ly/srm2019lab](http://bit.ly/srm2019lab)** in folder `code`.

## Part 1: Basic MQTT

In this section you will work with MQTT using the [**MQTT Paho library**](https://www.eclipse.org/paho/clients/python/) for Python. The documentation of the MQTT Paho API is [here](https://www.eclipse.org/paho/clients/python/docs/). You have to first install it using:

$ sudo pip3 install paho-mqtt

In this session we will use a **public broker**. There are various public brokers (also called `sandboxes`) in the Internet. For example:
* `iot.eclipse.org` (https://iot.eclipse.org/getting-started#sandboxes)
* `test.mosquitto.org` (http://test.mosquitto.org/)
* `broker.hivemq.com` (https://www.hivemq.com/public-mqtt-broker/)

An alternative **local** broker is this: `mqttbroker.grc.upv.es`

A more or less updated list is available here: https://github.com/mqtt/mqtt.github.io/wiki/public_brokers

:::info
NOTE: in case you are interested, it's very easy to install one's own broker (see for example https://mosquitto.org/download/).
:::

### A simple subscriber

File: `sisub.py` cointains the code of a simple python subscriber. This code connects to a public broker and subscribes to topic `$SYS/#`.

**Take a look at the code and try to understand it.**

:::danger
1.- Ejecuta este codigo y explica el resultado obtenido.
:::

### A simple producer

File: `sipub.py` cointains the code of a simple python producer. This code connects to a public broker and periodically publishes random values to topic `"PMtest/rndvalue"`

**Take a look at the code and try to understand it.**

:::danger
2.- Ejecuta este codigo y explica el resultado obtenido.
:::

:::danger
3.- Modifica `sisub.py` para poder recibir los datos enviados por `sipub.py`.
:::

### Some basic tests

#### One to many communication

MQTT is very useful when we have to distribute the same piece of information to various users... but we have to be careful with topics.

For example, using the code from Question_3, set as ``topic`` for the "subscriber" the value ``i/LOVE/Python`` and execute it.

Now modify the code of the "producer" so that it uses the same topic (i.e., ``i/love/python``) and as the message use the text that you want. Execute the "producer".

:::danger
4.- Describe lo que obtienes y por que.
:::

#### Retained messages:
Normally, if a publisher publishes a message to a topic, and **no one is subscribed** to that topic the message is simply discarded by the broker. If you want your broker to remember the last published message, you'll have to use the ``retained`` option.
Only one message is retained per topic. The next message published on that topic replaces the retained message for that topic.

Modify the `loop_start/loop_stop` section of `sipub.py` like indicated below:

client.loop_start()

msg_to_be_sent = "__whatevertextyouwant__"
client.publish(THE_TOPIC, 
               payload=msg_to_be_sent, 
               qos=0, 
               retain=False)

client.loop_stop()

Try now the following cases, but **remember to always execute the "subscriber" AFTER the "producer"** in all cases:

:::danger
5.- Prueba los siguentes casos:
* Publica un mensaje con la opcion de retained a "False". ¿Qué recibe el "subscriber"?
* Publica un mensaje con la opcion de retained a "True". ¿Qué recibe el "subscriber"?
* Publica varios mensajes (diferentes) con la opcion de retained a "True" antes de activar el "subscriber". ¿Qué recibe el "subscriber"?
:::

Finally, how do you remove or delete a retained message? You have to publish a blank message with the retain flag set to true. ==Try it.==

:::danger
6.- Crea grupos de dos o tres miembros. Crea una aplicación de chat muy básica, donde todos los mensajes publicados de cualquiera de los miembros sean recibidos **solo por los miembros** del grupo.
> la lectura de texto desde el teclado en Python se puede hacer usando:
> `name = input("Enter text: ")`
:::

## Part 2: Getting started with IoT devices

In this part you will learn how to use a IoT devices. The HW you will use is a [LoPy4](https://pycom.io/product/lopy4/) with a [PySense](https://pycom.io/product/pysense/) board.


LoPys are based on [MicroPython](https://micropython.org/). MicroPython is a full Python compiler and runtime that runs on the bare-metal. You get an interactive prompt, called the REPL ("Read Evaluate Print Loop"), to execute commands immediately, along with the ability to run and import scripts from the built-in filesystem. The REPL has history, tab completion, auto-indent and paste mode.
More infos about LoPy's REPL can be found here: https://docs.pycom.io/pymakr/toolsfeatures/

To connect to the LoPy's REPL you will use a Python tool called **[mpy-repl-tool](https://mpy-repl-tool.readthedocs.io/en/latest/)**. You will learn how to use it in this section.

==Now, plug-in your LoPy device in a USB port and wait a few seconds until the LED starts blinking in blue. Before, you'll probably be asked whether you want to use the device in the VM==


Then execute:

python3 -m there detect

This command will list the serial ports and "hopefully" :smiley: will automatically find your LoPy board. If everything is fine you should see something like:

/dev/ttyACM0 - Pysense - Pysense

To get access to the REPL prompt, write:
* `python3 -m there -i`

You'll get the `>>>` REPL prompt (see below). To exit from the REPL you have to type `Ctrl+]`


### The basic commands:

These are the basic commands that you will have to use:

* getting a list of the files on the LoPy:

$ python3 -m there ls -l /flash/*
----------    0    0     34B 1980-01-01 01:09:16 /flash/main.py
----------    0    0     29B 1980-01-01 01:09:18 /flash/boot.py

:::info
The filesystem has ``/`` as the root directory and the available physical drives are accessible from here. They are:
* `/flash`: the internal flash filesystem
* `/sd`: the SD card (if it is inserted, obviously)
:::

* coping file `_filename_` from your computer to the LoPy:

* wildcards can be used to copy multiple files from your computer to the LoPy. For example:

* reading the contents of file `_filename_` on the LoPy:

* removing file `_filename_` on the LoPy:

wildcards can be used to delete multiple files.

A detailed list of commands can be found here: [mpy-repl-tool.readthedocs](http://mpy-repl-tool.readthedocs.io/en/latest/commandline.html)

:::info
BTW; it can be useful to create the alias: `$ alias pyt="python3 -m there"` so not to type too much...
:::

### Typical Lopy usage workflow
A typical workflow is the following:

1. Write the micropython code in your computer
1. Copy the code file or files from your computer to the LoPy, e.g.,:

1. Start a serial terminal and get access to the REPL:

1. Execute the code in file `_filename_.py` on the LoPy, with:
`>>> import _filename_`

==Some useful commands are available [in the Appendix of this doc.](#Appendix-Just-in-case)==

### Let's see if you got it....

Execute the code in file `"led_blink.py"` on your LoPy.

:::danger
7.- indica los pasos que has tenido que seguir para ejecutar este codigo en la LoPy. ¿Cual es el resultado de la ejecucion de este codigo?
:::

### Accessing the sensors of the Pysense board

The PySense offers various embedded sensors, namely:
* Ambient light sensor
* Barometric pressure sensor
* Temperature sensor
* Humidity sensor
* 3 axis 12-bit accelerometer

Take a look at the code in file `"reads.py"` to understand how to use them. As you can see, various python libraries are necessary, they are all in the `lib` folder. To copy the whole `lib` folder to your LoPy, you have to execute:

:::danger
8.- indica los pasos que has tenido que seguir para ejecutar el file `"reads.py"` en la LoPy. ¿Cual es el resultado de la ejecucion de este codigo?
:::

---

## Part 3: MQTT clients with MicroPython and the LoPy

In this part you will experiment with MQTT using **MicroPython**. You will work with the **MicroPython** version of the code for a MQTT publisher and subscriber.

==You should already have copied the whole `lib` folder to your LoPy, by doing: `python3 -m there push -r lib/* /flash/lib`==

### A simple MicroPython subscriber

The code in file `mp_sisub.py` represents a simple subscriber. In this case we use the broker `iot.eclipse.org` but you can use any other accessible broker. 

**Take a look at the code and try to understand how it works.**

:::warning
Remember to first properly assign a value to **`wifi_user`** and **`wifi_passwd`**, using [this list](https://hackmd.io/@pmanzoni/SJyXHYHqr). 
:::

You can also, if you want, assign a value to `CLIENT_ID`. 

:::danger
9.- modifica `sipub.py` para enviar mensajes a este subscriber en la LoPy.
:::

### A simple MicroPython publisher

The code in file `mp_sipub.py` represents a simple publisher. In this case we use the broker `iot.eclipse.org` but you can use any other accessible broker. 

The code produces some random data and publish them. Take a look at the code below and try to understand how it works.


:::warning
Remember to properly assign a value to **`wifi_user`** and **`wifi_passwd`**, using [this list](https://hackmd.io/@pmanzoni/SJyXHYHqr). 
:::

You can also, if you want, assign a value to `CLIENT_ID`. 

:::danger
10.- Modifica `sisub.py` para recibir los mensajes que genera la LoPy con `mp_sipub.py`. ¿Que cambios has tenido que hacer?
:::


## Part 4: Sending data to a cloud based platform


In this block you will experiment about how MQTT can be used to send data to a cloud based platform. This procedure allows you to store your data in a cloud based repository and to analyze your data with software tools made available by the used platform. 

The cloud based platform you will use is called **Ubidots**. You will have to first create your free account in the Ubidots platform here: https://ubidots.com/stem/

Then you have to add a **Device** (select first the "Devices" section in the top on the web page):
![](https://i.imgur.com/cmCWh9I.png =x200)
choose:
![](https://i.imgur.com/M7nUX31.png =x100)

and add a name, like:
![](https://i.imgur.com/Bry2Axc.png =x200)

Now click on the device name and you'll get to the variables creation section:
![](https://i.imgur.com/0IXjq4Q.png =x200)

click on "Add Variable" and create a "Raw" type variable:
![](https://i.imgur.com/fRrfL6O.png =x200)


### Sending data to Ubidots using MQTT with Python

Now you will send data to our device **using MQTT with Python, that is not yet with the LoPy**. Take a look first to the Ubidots MQTT API Reference: https://ubidots.com/docs/hw/?language=Python#mqtt


The name of the broker for educational users is **"things.ubidots.com"**. To interact with it, you will need a TOKEN. To get yours click on “API Credentials” under your profile tab:

![](https://i.imgur.com/StmHEVg.png)

In my case I have:

![](https://i.imgur.com/pCmKyeQ.png)

To connect to the MQTT broker you'll have to use your ``Default Token`` as the MQTT username, and `None` as password.

The **topic** you have to use is **`/v1.6/devices/{LABEL_DEVICE}/{LABEL_VARIABLE}`** where you have to replace the ``API labels`` `{LABEL_DEVICE}` (e.g., mydevice_pm) and `{LABEL_VARIABLE}`  (e.g., the_varpm). 


![](https://i.imgur.com/QKfFnzp.png =200x)


The data must be represented using JSON. The simplest format is: `{"value":10}` 
> the string ``value`` is fixed and defined by the Ubidots API


So, summing up, to send value 25 to variable `the_varpm` of device ``mydevice_pm`` the code should look like:  

...
msg_to_be_sent = '{"value":25}'
client.publish("/v1.6/devices/mydevice_pm/the_varpm",
payload=msg_to_be_sent,
qos=0,
retain=False)
...

You'll get:

![](https://i.imgur.com/YsboDek.png)

:::danger
11.- Repite los pasos anteriores con tus datos de Ubidots (credenciales y nombre del dispositivo y variable usados)  y utilizando el codigo del  "producer" (`sipub.py`).
:::

### Sending data from the LoPy

:::danger
12.- Repite los pasos anteriores enviando los datos desde la LoPy, utilizando como referencia el codigo de `mp_sipub.py`. Puedes enviar cualquiera de los datos que la Pysense te ofrece... mira el codigo `reads.py` que has utilizado antes para leer datos de los sensores de la LoPy.

Para conectarte al broker tienes que utilizar:
`client = MQTTClient(dev_id, broker_addr, 1883, user=TOKEN, password='None')`

:::

:::info
Para manejar los datos JSON, python ofrece esta clase:

>>> import json
>>> d = {'sensorId': 'temp1', 'Value': 25}
>>> dj = json.dumps(d)
>>> nd = json.loads(dj)

:::



### Creating dashbord elements.

Ubidots allows to visualize the data in various graphical ways.

Go to the Dashboard section and add new widget associated with the variable of your device.
![](https://i.imgur.com/YDQBZ4z.png)

See the alternatives you have and how they can be configured.

:::danger
13.- Haz una captura de pantalla de la dashboard que has creado
:::

---

# Appendix: Just in case

LoPy's documentation is here: https://docs.pycom.io/index.html


## Problem "pushing" a file to the LoPy
Sometime you get an error message when "push"ing a file to the LoPy, something like this:

$ python3 -m there push led_blink.py /flash
ERROR: action or command failed: execution failed: : Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: _h

the solution is to use the **`--force`** option. That is:

$ python3 -m there push --force led_blink.py /flash

## Resetting the LoPy
They are different ways to reset your device. Pycom devices support both soft and hard resets. 

A **soft reset** clears the state of the MicroPython virtual machine but leaves hardware peripherals unaffected. To do a soft reset, press `Ctrl+D` while in the REPL.

A **hard reset** is the same as performing a power cycle to the device. In order to hard reset the device, press the reset switch or run:

>>> import machine
>>> machine.reset()