# Introduction project IOT
Tobias Mesch tm222mu
**About project**
---
Learning the process of IOT. From using a sensor with a processorkit then complicate things further. Get the data from sensor. Transmitting the data of this sensor. Then present this data.
**Time invested**
---
Due to hard learning curve, lots of time got invested into reaserching alot.
Installing firmware ~5min.
Assembling the device ~5-10 min.
Inject code into device and writing it. ~15min.
Fire up Ubidots and manage the data. ~10min.
An hour or so should be sufficient to make the same steps as I did. Reaserch is hard to put a number on.
**Objective**
---
Why you chose the project
I wanted to learn alot more about IOT and all behind the scenes parts.
What purpose does it serve
1. Setting up the pycom with a sensor
2. Code python to gather the data from sensor
3. Grab a protocol for transfering data
4. Retrive the data on the other end of cloud
What insights you think it will give
More inspiriation to do further advanced project that might have more relevance of the everyday living. How easy and hard some parts might be. What obstacles needed to tackle.
**Material**
---
List of material:
I bought the bundle from electrokit (995kr), this contained:
Lopy4 - https://pycom.io/product/lopy4/
Expansionboard 3.1 (not 3.0) - https://pycom.io/product/expansion-board-3-0/
Antenna - https://pycom.io/product/lora-868mhz-915mhz-sigfox-antenna-kit/
Sensorkit - https://www.electrokit.com/produkt/sensor-kit-26-moduler/
Wires - included in sensorkit
Breadboard (59kr) - https://www.electrokit.com/produkt/kopplingsdack-400-anslutningar/
Li-Po Battery (99kr) - https://www.kjell.com/se/produkter/el-verktyg/arduino/tillbehor/luxorparts-li-po-batteri-37-v-med-kontakt-1200-mah-p87924
For this particular project only one sensor is used from the kit.
So pricing should fall down if only one sensor is bought and battery is optional so:
lopy4+expansionboard+antenna and 3 wires from female - male.
Should be roughly 600kr.
Expansionboard is like the motherboard with other functional properties.
Lopy4 gives access to LoRa, Sigfox, WiFi or Bluetooth.
Antenna for sending and recieving signals.
Wires to connect sensor with the lopy.
Sensor to read data from environment.
Battery to take the kit on a wild adventure.
I got my own pc with windows10 (do not require my computer to do the same stuff :) )
**Computer setup**
---
Chosen IDE
Atom with pymkr plugin. https://atom.io/ + https://docs.pycom.io/pymakr/installation/atom/
How the code is uploaded
Code is transfer over USB cable. Using COM4 port.
Steps that you needed to do for your computer. Installation of Node.js, extra drivers, etc.
This took some time to get ready. First download all packages needed according to pycom https://docs.pycom.io/pytrackpysense/installation/firmware/.
Packages downloaded:
* dfu-util v0.9(.zip) - http://dfu-util.sourceforge.net/releases/dfu-util-0.9-win64.zip
* expansion31_0.0.11.dfu - https://software.pycom.io/findupgrade?key=expansion31.dfu&type=all&redirect=true
* Zadiq - http://zadig.akeo.ie/
* pycomFW - https://software.pycom.io/findupgrade?product=pycom-firmware-updater&type=all&platform=win32&redirect=true
Put all these in one folder for easier management.
First I started Zadiq. Changed option -> list all devices. If working correct I can see all my USB devices. And what driver are installed. USB ID shows manufacture ID and device ID. A dropbox for what should be updated we can search for install driver, wcid or extract firmware from selected device.
I plugged the Expansionboard into the usb connection. Press the 's1' and hold this. Then end of usb-cable into the computer. Wait for a second. Released button. Select device that matches my own. Name was 'unknown 1'. But manufacture ID and device ID matched. 04D8 EF99 If in DFU bootload mode, else it was shown 04D8 EF98. Took some retrys since I missed bootload menu. In 7s timeframe I need to start doing stuff on zadiq. libusbk(v3.0.7.0) Install driver. If already found, reinstall driver. If replaced correctly 'successful DFU driver installation' or an error message. Somehow, if this is done but same driver already exists I got an error message. But reading the log 0x0 would be displayed at bottom meaning no errors. After this, lets go to device manager or in swedish enhetshanterare. Lets search in the list for libusbK USB Devices or COMports. I found one with the name expansion3. So It seems correct so far. However, when I tried to go for next step for uploading the expansion.dfu into the board I didnt get a usb connection. 'No DFU capable USB device available'. Lets go back to device manager. Aha. Port(COM & LPT) is missing. Dont think everything went correct in installation, or I messed up. Lets go to Zadiq. Grab the expansion3 device. Change -> USB Serial (CDC). Lets install this. Magic appears and now expansion3 moved from libusbk into port/com4. I could now talk to my board.
**DeviceManager:**
Correct connection when in application firmware.
Connection looks like this for me in DFU bootloader.
When no connection exist libusbK or Expansion3 doesnt show.
**Zadiq:**
Expansion3->Application firmware. Check usbID on both this and the one below. EF98,EF99. Tells us what mode the device was booted in. Driver should be Usbser(v1.0.0.0) otherwise press 'Replace' with desired driver.
Unkown->bootload. Correct driver shown on left side, to replace shift up or down arrow then press 'Replace'. libusbK(v3.0.7.0) is correct one for me.
Next part, open CMD in admin mode. cd into correct directory of all my files ive gathered. Run the command. dfu-util-static.exe -D expansion31_0.0.11.dfu. End lines should be of: Download done.
state(2) = dfuIDLE, status(0) = No error condition is present
Done!. This is great. It worked.
Lets unplug the usb from the computer.
(sorry no photo from this part)
Continue with Lopy4 - https://docs.pycom.io/gettingstarted/connection/lopy4/
Mount lopy on the expansionboard if lazy. If not lazy solder all the wires around as pleased.
Take antenna wire and plug onto the lopy as seen in schematic. 3 spots, but only one is correct. Top left.
Take antenna and screw down the connector. Now the antenna should be attached to wire that is attached to lopy that is attached to expansionboard.
Lets grab pycom firmware for computer. https://docs.pycom.io/gettingstarted/installation/firmwaretool/
Since firmware on board is already updated skip 1. Skip 2 since device is unplugged. 3, module is on. Reconnect the usb cable to computer. Run the firmware. Pycom own wizard should appear. Not to be trusted according to my computer. But I think its a good precaution. Override this and keep installing. Ill install the latest version 1.16.2[stable]. Do not include stuff or send statistics. Select COM4, since this is expansionboard connection. High speed transfer. Advanced settings. All else is default. Then continue installing. If no errors, all good. I didnt get any errors this far down. So unplug device, and plug it in again. Should be ready to use.
From here I could go on with Atom, install this, and then install the pymakr plugin. See pycom guide https://docs.pycom.io/gettingstarted/installation/pymakr/. Quite self explaining guide.
This was taken from documentation https://www.electrokit.com/produkt/digital-temperatur-och-fuktsensor-dht11/
MCU on the photo is connected to our pycom for transmitting data. Hooked onto pin P23. VDD set to 3V according to documentation (3-5.5V). GND to 0. (No pull-up or pull-down resistors, if data shown was bad I might look into this, but data seems fine).
According to lopy4 docs https://docs.pycom.io/gitbook/assets/specsheets/Pycom_002_Specsheets_LoPy4_v2.pdf.
Idle power consumption: 30mA
Wifi transmitting: 107mA
Deepsleep: 19.5mA
The sensor has average of 1mA.
In code alot of reading and sending is done (not optimal). Thus a very high consumption. (look at ubidots charts. Running on fullycharged battery ~8h).
**Platform**
---
Ubidots own platform https://ubidots.com/stem/
Free non-commercial license for students, free is best, even if its only for a month.
Easy to setup with pycom, lots of documentation and gave all the functionality I needed. Since Im rookie with no experience in this field I wanted an easy setup, with lots of functionality.
A route I tried was going trough pybytes own portal, and transmit from them to a webhookserver. But then I needed to transfer data from there to another site, or down into database and then fire up to a visualizationProgram. Far to many steps for my taste. If this was a buisness that would been a plausible route. Or send the data to own server to store, in say mySQL or simple textfile since only one sensor is active. From there gather data into Grafana or other vizualization. Locally would probably give more security and integrity over the data (also spend more money for this service). If im scaling I probably hook the data into own server over the ethernet. Then have a webservice or app to stream data to paying customers or research purposes.
**The code**
---
Sensor code was taken from https://github.com/iot-lnu/applied-iot-20/tree/master/sensor-examples/DHT11-22
Ubidots connection follow this https://help.ubidots.com/en/articles/961994-connect-any-pycom-board-to-ubidots-using-wi-fi-over-http
Added these 2 togheter and modified it a bit myself.
(I shouldve made a github for this code section, ops)
Main
```python
from network import WLAN
import urequests as requests
import time
import machine
from machine import Pin
import _thread
from dht import DHT
import math
TOKEN = "insertTokenPlease" #Put here your TOKEN https://help.ubidots.com/en/articles/590078-find-your-token-from-your-ubidots-account
DELAY = 1 # Delay in seconds
wlan = WLAN(mode=WLAN.STA)
wlan.antenna(WLAN.INT_ANT)
# Assign your Wi-Fi credentials
wlan.connect("SSIDorWifuName", auth=(WLAN.WPA2, "uberHardPassword"), timeout=5000)
while not wlan.isconnected ():
machine.idle()
print("Connected to Wifi\n")
# Builds the json to send the request
def build_json(variable1, value1, variable2, value2, variable3, value3):
try:
lat = 59.2350 #set latitude
lng = 19.2288 #set longitude
data = {variable1: {"value": value1},
variable2: {"value": value2, "context": {"lat": lat, "lng": lng}},
variable3: {"value": value3}}
return data
except:
return None
print('def build_json')
# Sends the request. Please reference the REST API reference https://ubidots.com/docs/api/
def post_var(device, value1, value2, value3):
try:
url = "https://industrial.api.ubidots.com/"
url = url + "api/v1.6/devices/" + device
headers = {"X-Auth-Token": TOKEN, "Content-Type": "application/json"}
data = build_json("temperature", value1, "position", value2, "humidity", value3)
if data is not None:
print(data)
req = requests.post(url=url, headers=headers, json=data)
return req.json()
else:
pass
except:
pass
print('def post_var')
# Type 0 = dht11 <- my choise
# Type 1 = dht22
th = DHT(Pin('P23', mode=Pin.OPEN_DRAIN), 0) #creates the thread that reads from pin23
time.sleep(2)
i = 0 #counter
iThreshold = 12 #value for the counter , 12 for every min
temp=[] #list for temperature
humid=[] #list for humidity
while True:
result = th.read() #get result from thread
while not result.is_valid(): #check if result is non-valid
time.sleep(.5) #sleep half a sec so reading from device is not overkill :)
result = th.read() #get result from thread again
temp.append(result.temperature)#append list with result
humid.append(result.humidity)#append list with result
#prints to console
print('i:',i) #get the counter
print('Temp:', result.temperature) #get the temperature in Celcius
print('RH:', result.humidity) #get the humidity %
if i>iThreshold: #if counter passes the threshold do action
i=0 #reset counter
avrTemp = sum(temp)/len(temp) #get avr
avrhumid = sum(humid)/len(humid) #get avr
post_var("pycom", avrTemp,1,avrhumid) #post values
temp = [] #reset list for temperature
humid = [] #reset list for humidity
print("values send")
print("=====")
i=i+1 #increase counter
time.sleep(5) #sleep 5 seconds, go read values again
#sending signals every min, due to the sleep
print("Goodbye") #probably something went wrong and it is shutting down
```
lib/dht
```python
import time
from machine import enable_irq, disable_irq
class DHTResult:
'DHT sensor result returned by DHT.read() method'
ERR_NO_ERROR = 0
ERR_MISSING_DATA = 1
ERR_CRC = 2
error_code = ERR_NO_ERROR
temperature = -1
humidity = -1
def __init__(self, error_code, temperature, humidity):
self.error_code = error_code
self.temperature = temperature
self.humidity = humidity
def is_valid(self):
return self.error_code == DHTResult.ERR_NO_ERROR
class DHT:
'DHT sensor (dht11, dht21,dht22) reader class for Pycom'
# __pin = Pin('P3', mode=Pin.OPEN_DRAIN)
__dhttype = 0
def __init__(self, pin, sensor=0):
self.__pin = pin
self.__dhttype = sensor
self.__pin(1)
time.sleep(1.0)
def read(self):
# time.sleep(1)
# send initial high
# self.__send_and_sleep(1, 0.025)
# pull down to low
self.__send_and_sleep(0, 0.019)
# collect data into an array
data = self.__collect_input()
# print(data)
# parse lengths of all data pull up periods
pull_up_lengths = self.__parse_data_pull_up_lengths(data)
# if bit count mismatch, return error (4 byte data + 1 byte checksum)
# print(pull_up_lengths)
# print(len(pull_up_lengths))
if len(pull_up_lengths) != 40:
return DHTResult(DHTResult.ERR_MISSING_DATA, 0, 0)
# calculate bits from lengths of the pull up periods
bits = self.__calculate_bits(pull_up_lengths)
# we have the bits, calculate bytes
the_bytes = self.__bits_to_bytes(bits)
# print(the_bytes)
# calculate checksum and check
checksum = self.__calculate_checksum(the_bytes)
if the_bytes[4] != checksum:
return DHTResult(DHTResult.ERR_CRC, 0, 0)
# ok, we have valid data, return it
[int_rh, dec_rh, int_t, dec_t, csum] = the_bytes
if self.__dhttype == 0: # dht11
rh = int_rh # dht11 20% ~ 90%
t = int_t # dht11 0..50°C
else: # dht21,dht22
rh = ((int_rh * 256) + dec_rh)/10
t = (((int_t & 0x7F) * 256) + dec_t)/10
if (int_t & 0x80) > 0:
t *= -1
return DHTResult(DHTResult.ERR_NO_ERROR, t, rh)
def __send_and_sleep(self, output, mysleep):
self.__pin(output)
time.sleep(mysleep)
def __collect_input(self):
# collect the data while unchanged found
unchanged_count = 0
# this is used to determine where is the end of the data
max_unchanged_count = 100
last = -1
data = []
m = bytearray(800) # needs long sample size to grab all the bits from the DHT # noqa
irqf = disable_irq()
self.__pin(1)
for i in range(len(m)):
m[i] = self.__pin() # sample input and store value
enable_irq(irqf)
for i in range(len(m)):
current = m[i]
data.append(current)
if last != current:
unchanged_count = 0
last = current
else:
unchanged_count += 1
if unchanged_count > max_unchanged_count:
break
# print(data)
return data
def __parse_data_pull_up_lengths(self, data):
STATE_INIT_PULL_DOWN = 1
STATE_INIT_PULL_UP = 2
STATE_DATA_FIRST_PULL_DOWN = 3
STATE_DATA_PULL_UP = 4
STATE_DATA_PULL_DOWN = 5
state = STATE_INIT_PULL_UP
lengths = [] # will contain the lengths of data pull up periods
current_length = 0 # will contain the length of the previous period
for i in range(len(data)):
current = data[i]
current_length += 1
if state == STATE_INIT_PULL_DOWN:
if current == 0:
# ok, we got the initial pull down
state = STATE_INIT_PULL_UP
continue
else:
continue
if state == STATE_INIT_PULL_UP:
if current == 1:
# ok, we got the initial pull up
state = STATE_DATA_FIRST_PULL_DOWN
continue
else:
continue
if state == STATE_DATA_FIRST_PULL_DOWN:
if current == 0:
# we have the initial pull down,
# the next will be the data pull up
state = STATE_DATA_PULL_UP
continue
else:
continue
if state == STATE_DATA_PULL_UP:
if current == 1:
# data pulled up, the length of this pull up
# will determine whether it is 0 or 1
current_length = 0
state = STATE_DATA_PULL_DOWN
continue
else:
continue
if state == STATE_DATA_PULL_DOWN:
if current == 0:
# pulled down, we store the length of
# the previous pull up period
lengths.append(current_length)
state = STATE_DATA_PULL_UP
continue
else:
continue
return lengths
def __calculate_bits(self, pull_up_lengths):
# find shortest and longest period
shortest_pull_up = 1000
longest_pull_up = 0
for i in range(0, len(pull_up_lengths)):
length = pull_up_lengths[i]
if length < shortest_pull_up:
shortest_pull_up = length
if length > longest_pull_up:
longest_pull_up = length
# use the halfway to determine whether the period it is long or short
halfway = shortest_pull_up + (longest_pull_up - shortest_pull_up) / 2
bits = []
for i in range(0, len(pull_up_lengths)):
bit = False
if pull_up_lengths[i] > halfway:
bit = True
bits.append(bit)
return bits
def __bits_to_bytes(self, bits):
the_bytes = []
byte = 0
for i in range(0, len(bits)):
byte = byte << 1
if (bits[i]):
byte = byte | 1
else:
byte = byte | 0
if ((i + 1) % 8 == 0):
the_bytes.append(byte)
byte = 0
# print(the_bytes)
return the_bytes
def __calculate_checksum(self, the_bytes):
return the_bytes[0] + the_bytes[1] + the_bytes[2] + the_bytes[3] & 255
```
lib/urequest
```python
import usocket
class Response:
def __init__(self, f):
self.raw = f
self.encoding = "utf-8"
self._cached = None
def close(self):
if self.raw:
self.raw.close()
self.raw = None
self._cached = None
@property
def content(self):
if self._cached is None:
try:
self._cached = self.raw.read()
finally:
self.raw.close()
self.raw = None
return self._cached
@property
def text(self):
return str(self.content, self.encoding)
def json(self):
import ujson
return ujson.loads(self.content)
def request(method, url, data=None, json=None, headers={}, stream=None):
try:
proto, dummy, host, path = url.split("/", 3)
except ValueError:
proto, dummy, host = url.split("/", 2)
path = ""
if proto == "http:":
port = 80
elif proto == "https:":
import ussl
port = 443
else:
raise ValueError("Unsupported protocol: " + proto)
if ":" in host:
host, port = host.split(":", 1)
port = int(port)
try:
ai = usocket.getaddrinfo(host, port, 0, usocket.SOCK_STREAM)
except:
ai = usocket.getaddrinfo(host, port)
s = usocket.socket()
ai = ai[0]
try:
s.connect(ai[-1])
if proto == "https:":
s = ussl.wrap_socket(s, server_hostname=host)
s.write(b"%s /%s HTTP/1.0\r\n" % (method, path))
if not "Host" in headers:
s.write(b"Host: %s\r\n" % host)
# Iterate over keys to avoid tuple alloc
for k in headers:
s.write(k)
s.write(b": ")
s.write(headers[k])
s.write(b"\r\n")
if json is not None:
assert data is None
import ujson
data = ujson.dumps(json)
s.write(b"Content-Type: application/json\r\n")
if data:
s.write(b"Content-Length: %d\r\n" % len(data))
s.write(b"\r\n")
if data:
s.write(data)
l = s.readline()
#print(l)
l = l.split(None, 2)
status = int(l[1])
reason = ""
if len(l) > 2:
reason = l[2].rstrip()
while True:
l = s.readline()
if not l or l == b"\r\n":
break
#print(l)
if l.startswith(b"Transfer-Encoding:"):
if b"chunked" in l:
raise ValueError("Unsupported " + l)
elif l.startswith(b"Location:") and not 200 <= status <= 299:
raise NotImplementedError("Redirects not yet supported")
except OSError:
s.close()
raise
resp = Response(s)
resp.status_code = status
resp.reason = reason
return resp
def head(url, **kw):
return request("HEAD", url, **kw)
def get(url, **kw):
return request("GET", url, **kw)
def post(url, **kw):
return request("POST", url, **kw)
def put(url, **kw):
return request("PUT", url, **kw)
def patch(url, **kw):
return request("PATCH", url, **kw)
def delete(url, **kw):
return request("DELETE", url, **kw)
```
boot
```python
from machine import UART
import machine
import os
uart = UART(0, baudrate=115200)
os.dupterm(uart)
machine.main('main.py')
```
**Transmitting the data / connectivity**
---
Since Lora Gateways was out of range from my own home, I couldnt use this solution (if i didnt build my own gateway. Future me would probably do this), I had to go for another option. Easy way was hooking into my wireless at home. So WiFi would be a good choise (powerconsumption will skyrocket thou).
Well since we already use a high power protocol, lets just chew up the ther power while at it. Every minute (roughly) it sends average data to ubidots. Every loop is on a sleeptimer 5s, then read data. (for low powerusage probably more wise to use lora, check every 30min or so and transmit then deepsleep. Used with MQTT aswell.)
The code packets data into json-objects. These are send over HTTP (or HTTPS if configured) over Wifu, then into big mist, then appears on Ubidots frontdoor and then their cloudserivce gives me access to the data.
**Presenting the data**
---
Pycom devices (easily made into diffrent, or by manipulating data from device, post_var("pycom", avrTemp,1,avrhumid) #post values)
Lets check how my battery went. Lots of data was gathered.
Lets check temperature. Timestamps and data. More than I thought was presented from my end.
Dashboard to present the data. Lots of diffrent choises. Easy to setup.
Lets look at the event tasks. Oh easy to setup, next next make some choises, next or next :)
Mail M------R! Oooh it works. I set the event to send 1 time if humidity was under 60 for a minute. Message recieved.
So much for free, and 30 days. Thats alot of features that probably would taken me a lifetime programming on my own.
**Finalizing the design**
---
Final thought. I wish more time was on my hands so I could setup a real lora gateway to test out the true power of this. Then hook up more sensors, more devices, more dataflow of all cind of stuff. This was superfun and so much was learned in this process. A 3d printer would also be next stop, to make the case for my device and put outdoors. Maybe even firing up own database and hook up a screen to have the live output of my own sensorfield. Now with lots of the basic done, probably more projects will be created on my own. A bit sad also, since this was the end of this time.
***Sorry for the long post here's a potato*** (according to Internet custom)
Sign in with Wallet
Connect another wallet