# MQTT Lab 1
> 2019 Workshop on Rapid Prototyping of IoT for Science (smr3268)
> http://bit.ly/ictp2019-lab1

This lab aims to offer you an hands-on experience with MQTT. You will perform experiments that will allow you to learn how to "publish" data and "subscribe" to get data.

**All the code necessary for this Lab session is available here:
http://bit.ly/ictp2019-mqtt_drive
in the "code" folder. You have to download it in your computer before starting the session.**
> file "code.zip" contains the whole directory and can be used to quickly download everything

# Section 1: Basic MQTT

In this section you will work with MQTT using the [**MQTT Paho** library](https://www.eclipse.org/paho/clients/python/) for Python. The documentation of the MQTT Paho API is [here](https://www.eclipse.org/paho/clients/python/docs/).

Programming will be done using a cloud development environment called [**repl.it**](https://repl.it/login). If you don't already have an account for it, you have to create one right now (it takes 3 seconds). repl.it will take care of installing the MQTT Paho library.

In this session we will use a **public broker** and access it using plain TCP (no websockets in this session).

## A simple subscriber

The code of the simplest subscriber is the one below.

## File: sisub.py

import paho.mqtt.client as mqtt

THE_BROKER = "test.mosquitto.org"
THE_TOPIC = "$SYS/#"
CLIENT_ID = ""

# The callback for when the client receives a CONNACK 
# response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected to ", client._host, "port: ", client._port)
    print("Flags: ", flags, "returned code: ", rc)

    client.subscribe(THE_TOPIC, qos=0)

# The callback for when a message is received from the server.
def on_message(client, userdata, msg):
    print("sisub: msg received with topic: {} and payload: {}".format(msg.topic, str(msg.payload)))

client = mqtt.Client(client_id=CLIENT_ID, 
                     clean_session=True, 
                     userdata=None, 
                     protocol=mqtt.MQTTv311, 
                     transport="tcp")

client.on_connect = on_connect
client.on_message = on_message

client.username_pw_set(None, password=None)
client.connect(THE_BROKER, port=1883, keepalive=60)

# Blocking call that processes network traffic, dispatches callbacks and
# handles reconnecting.
client.loop_forever()

This code connects to the broker `"test.mosquitto.org"` and subscribes to topic `"$SYS/#"`.

**Execute this code in repl.it and check what you get.**

## A simple producer

The code of the simplest producer is the one below.

# File: sipub.py

import random
import time

import paho.mqtt.client as mqtt

THE_BROKER = "test.mosquitto.org"
THE_TOPIC = "PMtest/rndvalue"
CLIENT_ID = ""

# The callback for when the client receives a CONNACK 
# response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected to ", client._host, "port: ", client._port)
    print("Flags: ", flags, "returned code: ", rc)

# The callback for when a message is published.
def on_publish(client, userdata, mid):
    print("sipub: msg published (mid={})".format(mid))

client = mqtt.Client(client_id=CLIENT_ID, 
                     clean_session=True, 
                     userdata=None, 
                     protocol=mqtt.MQTTv311, 
                     transport="tcp")

client.on_connect = on_connect
client.on_publish = on_publish

client.username_pw_set(None, password=None)
client.connect(THE_BROKER, port=1883, keepalive=60)

client.loop_start()

while True:

    msg_to_be_sent = random.randint(0, 100)
    client.publish(THE_TOPIC, 
                   payload=msg_to_be_sent, 
                   qos=0, 
                   retain=False)

    time.sleep(5)

client.loop_stop()

This code connects to the broker `"test.mosquitto.org"` and periodically publishes random values to topic `"PMtest/rndvalue"`

**Execute this code in repl.it and check how it works.
You will have to use the code of the "subscriber" of the previous block to check if it actually works (in a different window, obviously). What do you have to change in the code?**

## Some basic exercise

Now that we have a simple producer and a simple consumer (**don't close the two windows that you have in repl.it**), let's do some basic exercise.
We will actually repeat the examples of "Lab 0" but now with our own clients written in Python.

Let's start with a easy one. Use as ``topic`` for the "subscriber" the value ``i/LOVE/Python`` and execute it.

Now modify the code of the "producer" so that it uses the same topic (i.e., ``i/LOVE/Python``) and as the message use the text that you want. Execute the "producer".

What do you get now? You should get your message plus all the messages written by all the other clients in the room.
**With just one publish action you actually reached various devices!!**

### Retained messages:
Normally, if a publisher publishes a message to a topic, and *no one is subscribed* to that topic the message is simply discarded by the broker. If you want your broker to remember the last published message, you'll have to use the ``retained`` option.
Only one message is retained per topic. The next message published on that topic replaces the retained message for that topic.

Modify the `loop_start/loop_stop` section of the "producer" like indicated below:

client.loop_start()

msg_to_be_sent = "__whatevertextyouwant__"
client.publish(THE_TOPIC, 
               payload=msg_to_be_sent, 
               qos=0, 
               retain=False)

client.loop_stop()

Try now the following cases, but **remember to always execute the "subscriber", in all cases, AFTER** the "producer":
1. Publish a message with the retain message flag set to "False". What does the "subscriber" receives?
1. Publish a message with the retain message flag set to "True". What does the "subscriber" receives?
1. Publish several (different) messages with the retain message flag set to "True" before subscribing. What happens?
2. Publish a message with the retain message flag set to "False". What happens?

Finally, how do I remove or delete a retained message? You have to publish a blank message with the retain flag set to true. Try it.

### Are topics case-sensitive?

Modify now the ``topic`` field of the "producer" and set it to: ``i/love/python``, that is all lowercase, and as the message use the text that you want. Execute it.
**What happened? Did you receive any message? Why?**

## Exercise 1

Create groups of two or three devices. Create a very basic chat application, where all the messages published from any of the members are received **only by the members** of the group.
> reading text from keyboard in Python can be done using:
> `name = input("Enter text: ")`

## Exercise 2

Take a look at this web page http://ny-power.org/ and, by using MQTT, try to get some data about New York City energy consumption (see. http://ny-power.org/mqtt). For example, how much energy is produces using "Nuclear" sources??

# Section 2: Some warm-up experiments with the LoPys

The HW you will use are LoPy devices with a PySense board.


## Some basic code examples

Various python libraries are necessary for the following examples, they are all in the `lib` folder. Copy the whole `lib` folder to your LoPy.

### Example 1
Just to practice with the device, try to execute the code in file `"led_blink.py"` on your LoPy. **Is it working?**

### Example 2
The PySense offers various embedded sensors, namely:
* Ambient light sensor
* Barometric pressure sensor
* Temperature sensor
* Humidity sensor
* 3 axis 12-bit accelerometer

**Take a look and execute the code in file `"reads.py"` to understand how to use them.**

## MQTT clients with MicroPython and the LoPy

In this block you will see the **MicroPython** version of the code for a MQTT publisher and subscriber

Various MicroPython libraries are necessary for the following examples, they are all in the `lib` folder. Remember to copy the whole `lib` folder to your LoPy.

By the way:
* `ufun.py` provides the code to:

* `connect_to_wifi()`: connects the LoPy to a WiFi LAN. By properly passing the values `wifi_ssid` and `wifi_passwd` this function will try three times to connect to the specified AP, exiting if the operation is not possible.
* `random_in_range()`: generates random numbers in a range, and
* `set_led_to()` and `flash_led_to()`: simplify the control of the LED.

* `mqtt.py` the MQTT client library.

### A simple MicroPython subscriber

The code below represent a simple subscriber. In this case we use the broker `iot.eclipse.org` but you can use any other accessible broker. Take a look at the code below and try to understand how it works.

**Remember to properly assign a value to `THE_NAME_OF_THE_AP`** and, if you want, to `THE_TOPIC` and to `CLIENT_ID`.

# file: mp_sisub.py

from mqtt import MQTTClient
import pycom
import sys
import time

import ufun

wifi_ssid = 'THE_NAME_OF_THE_AP'
wifi_passwd = ''

THE_BROKER = "iot.eclipse.org"
THE_TOPIC = "test/SRM2018"
CLIENT_ID = ""

def settimeout(duration):
   pass

def on_message(topic, msg):
    print("Received msg: ", str(msg), "with topic: ", str(topic))

### if __name__ == "__main__":

ufun.connect_to_wifi(wifi_ssid, wifi_passwd)

client = MQTTClient(CLIENT_ID, THE_BROKER, 1883)
client.set_callback(on_message)

print ("Connecting to broker: " + THE_BROKER)
try:
	client.connect()
except OSError:
	print ("Cannot connect to broker: " + THE_BROKER)
	sys.exit()	
print ("Connected to broker: " + THE_BROKER)

client.subscribe(THE_TOPIC)

print('Waiting messages...')
while 1:
    client.check_msg()

Execute the code in the LoPy and **using the "producer" in repl.it** send some message to the LoPy.

### A simple publisher

Let's produce some random data using the code below. Take a look at the code below and try to understand how it works.

**Remember to properly assign a value to `THE_NAME_OF_THE_AP`** and, if you want, to `THE_TOPIC` and to `CLIENT_ID`.

# file: mp_sipub.py

from mqtt import MQTTClient
import pycom
import sys
import time

import ufun

wifi_ssid = 'THE_NAME_OF_THE_AP'
wifi_passwd = ''

THE_BROKER = "iot.eclipse.org"
THE_TOPIC = "test/SRM2018"
CLIENT_ID = ""

def settimeout(duration):
   pass

def get_data_from_sensor(sensor_id="RAND"):
    if sensor_id == "RAND":
        return ufun.random_in_range()

### if __name__ == "__main__":

ufun.connect_to_wifi(wifi_ssid, wifi_passwd)

client = MQTTClient(CLIENT_ID, THE_BROKER, 1883)

print ("Connecting to broker: " + THE_BROKER)
try:
    client.connect()
except OSError:
    print ("Cannot connect to broker: " + THE_BROKER)
    sys.exit()  
print ("Connected to broker: " + THE_BROKER)

print('Sending messages...')
while True:
    # creating the data
    the_data = get_data_from_sensor()
    # publishing the data
    client.publish(THE_TOPIC, str(the_data))
    print("Published message with value: {}".format(the_data))
    time.sleep(1)

Execute the code in the LoPy and **using the "consumer" in repl.it** get the messages produced by the LoPy.

# Section 3: interacting with a cloud platform

In this block you will experiment about how MQTT can be used to send data to a cloud based platform. This procedure allows you to store your data in a cloud based repository and to analyze your data with software tools made available by the used platform. For these experiments we will use the [Ubidots](https://ubidots.com/) platform.

You will have to first create your free account in the Ubidots platform here: https://app.ubidots.com/accounts/signup/

Then you have to add a **Device** (select first the "Devices" section in the top on the web page):

and then create a "Default" type variable:

Now we will send data to our device using MQTT. Take a look first to the Ubidots MQTT API Reference: https://ubidots.com/docs/hw/#mqtt

## Sending data to Ubidots using plain Python

We first use the MQTT "producer" in repl.it to send data to the ``variable`` of our Ubidots ``device``:

The name of the broker for educational users is **"things.ubidots.com"**. To interact with it, you will need a TOKEN. To get yours click on “API Credentials” under your profile tab:


In my case I have:


**IMPORTANT:**
**To connect to the MQTT broker you'll have to use your ``Default Token`` as the MQTT username, and `None` as password.**

Here:

The **topic** you have to use is **`/v1.6/devices/{LABEL_DEVICE}/{LABEL_VARIABLE}`** where you have to replace the ``API labels`` `{LABEL_DEVICE}` (e.g., srmdev) and `{LABEL_VARIABLE}` (e.g., temp).
For example:


The data must be represented using JSON. The simplest format is: `{"value":10}`
> the string ``value`` is fixed and defined by the Ubidots API

So, summing up, to send value 25 to variable `temp` of device ``srmdev`` the code should look like:

...
msg_to_be_sent = '{"value":25}'
client.publish("/v1.6/devices/srmdev/temp", 
                payload=msg_to_be_sent, 
                qos=0, 
                retain=False)
...

You'll get:


So try to repeat all the previous steps with your own device and variable.

## Sending data to Ubidots using the LoPys

We will now create a MicroPython periodic publisher that sends the temperature values sensed with your PySense to the variable you created in your Ubidots account.

Use the `lopy2ubi.py` code in the repository. Remember to properly assign the values of all the variables.

from mqtt import MQTTClient
import pycom
import sys
import time
import json
import ufun

from pysense import Pysense
from SI7006A20 import SI7006A20
import pycom
import micropython
import machine
import time

wifi_ssid = 'THE_NAME_OF_THE_AP'
wifi_passwd = ''
dev_id = 'NAME_YOUR_DEVICE'

broker_addr = 'THE_NAME_OF_THE_BROKER'
user_name = 'YOUR_UBIDOTS_USERNAME'
topic_is = "TOPIC_TO_BE_USED"


def settimeout(duration):
   pass

### if __name__ == "__main__":

py = Pysense()
tempHum = SI7006A20(py)

ufun.connect_to_wifi(wifi_ssid, wifi_passwd)

client = MQTTClient(dev_id, broker_addr, 1883, user=user_name, password='None')

print ("Connecting to broker: " + broker_addr)
try:
	client.connect()
except OSError:
	print ("Cannot connect to broker: " + broker_addr)
	sys.exit()	
print ("Connected to broker: " + broker_addr)

print('Sending messages...')
while True:
    temperature = tempHum.temperature()
    print("Temperature: {} Degrees".format(temperature))

    sensor_value = {'value': temperature}

    # publishing the data
    client.publish(topic_is, json.dumps(sensor_value))
    time.sleep(15)

Load the file `lab2main.py` to the LoPy and execute it. **Now check what happens in your Ubidots account. Are the data correctly arriving?**

# Exercises

## Ex. 1: more data

Extend the code above to publish to Ubidots the values of the other sensors available in the LoPy.

## Ex. 2: Creating dashbord elements.

Ubidots allows to visualize the data in various graphical ways.

Go to the Dashboard section and add new widgets associated with the variables of your device.

See the alternatives you have and how they can be configured.

## Ex. 3: Using events

Check also the possibilities offered by the **"Events" section** of the Ubidots

## Ex. 4: Playing with two brokers

Use the Raspberry Pi with the SenseHat to show on the "screen" one of the values that you are going to send to Ubidots...

Warning: you will have to use two brokers simultaneously