# Light and temperature data visualized
## Andreas Hortlund | ah225ba

## Project Overview
The project is made up so that you can measure light and temperature and send it to a server to then present the data visually. This guide is only sending data to an local instance of grafana and influxdb, but its easy to switch to a cloud instance of this.

The idea is to then be able to connect this further to maybe an air conditioner and electronic drapes, tho this is not covered in here due to resource shortage.

## Time estimate for the project
If you follow the guide the project will take about 2-4 hours depending on how familiar you are with the components.

## Objective of the project
The objective for this project was to learn the basics of IoT and get a better understanding of what it takes to develop for IoT-devices, also to learn more about working with embedded systems.

The purpose of the project is to get a C on the IoT summer course, the data will be able to show some correlation between the amount of light and the temperature over a long period. Other than that I don't think that it will give any revolutionary insights.

## Materials needed
List of materials and links to them.

| Material | Price |
|:-----------------------------------------------------------------------------------------------------:|:------:|
| [Lopy 4](https://pycom.io/product/lopy4/) | ~350kr |
| [Expansionboard](https://core-electronics.com.au/pycom-expansion-board-3-1.html) | ~400kr |
| [Breadboard](https://www.electrokit.com/en/product/solderless-breadboard-400-tie-points/) | 59kr |
| [Jumpwires](https://www.electrokit.com/en/product/hook-up-wires-w-pins-for-breadboard-smooth-65-pcs/) | 36kr |
| [10k ohm resistor](https://www.electrokit.com/en/product/resistor-metal-film-0-6w-1-10kohm-10k/) | 1kr |
| [TMP 36gz Thermistor](https://www.electrokit.com/en/product/tmp36-to-92-temperature-sensor/) | 19kr |
| [LDR](https://www.electrokit.com/en/product/photo-resistor-cds-2-5-kohm/) | 8kr |
| [USB to micro-USB cable](https://www.electrokit.com/en/product/usb-cable-a-male-microb-male-1-8m/) | 39kr |

### Lopy4
The lopy for with the protocols it support listed on it. This is the backbone of it all and runs micropython and have a live interpreter on it allowing you to run and upload python source files to it.

### Expansion board 3.1
The expansion board is there for extending the IO in the lopy4, giving you more connections option, being able to connect it the the computer and power it via an external battery.

### Breadboard
The breadboard is not needed, but is an awesome way to organize your sensors, their connection and power. Basically its cable management in a box.

### Jumpwires
The jumpwires is there to connect the sensors and lopy4/ expansion board to one another.

### 10k ohm resistor
This is needed for the LDR since the output from the lopy is to great for the LDR.

### TMP 36gz
This sensor is an thermistor that measures the temperature by increasing/decreasing the resistance in itself then you can calculate the temperature depending on the outgoing voltage.

### LDR
The LDR is an photoresistor that increases/decreases the resistance in itself depending on the amount of light it receives, in the same way in a thermistor we can then measure the output voltage.

### USB cable
Needed to connect the lopy4 with expansion board to a computer.

## Computer setup

### IDE
To be able to code and upload the code to the lopy4 i have chosen to go with and IDE called [visual studio code](https://code.visualstudio.com/). I choose it because of its portability, its lightweight and support multiple of plugins, of one called pymakr, that can be found [here](https://marketplace.visualstudio.com/items?itemName=pycom.Pymakr).

You can of course also search for it in the IDE itself and install it from there.

After you install and enable the plugin you will get access to the "REPL" that stands for "Read Evaluate Print Loop" that's the name of the micropython interpreter, you can think of it like a lightweight python interpreter. Because that's exactly what it is.


### Upload code
The code is uploaded via the upload button seen in the toolbar in the picture above. Visual studio code uploads the entire workspace so make sure you have main.py where the main code will run, and put any libraries in an folder named "lib". When the code is uploaded the lopy4 will reboot and run the new code directly.

### Updating firmware

For updating the firmware on the lopy4 with ease we will need to download pycoms update tool that can be found [here](https://pycom.io/downloads/) choose your platform of choice and download.

The same thing is needed for the expansion board, pycom has an awesome guide for this, check it out [here](https://docs.pycom.io/pytrackpysense/installation/firmware/)

But for the lopy4 the steps below will do the trick.


Choose the pybytes firmware for easier integration to the pybytes platform, tho we will not use it here, its good for testing things out with ease.

You can skip this step and follow the steps on [pybytes](https://pybytes.pycom.io/) if you wish to connect to that platform.

That's it! New firmware, no problems!

## Putting everything together

The circuit below is only for development purposes and should be better thought out and placed if planing to use it in production. The red and black wire is power respective ground. The brown and pink is in or "data" where we can measure voltage back to the lopy via ADC channels created in the code, take notice that the match the pins in the code, they are 15 and 16.


## Platform

For the platform I choose to go with a local docker instance running ubuntu with the services grafana, influxdb and telegraf installed and configured. I think this is an easy way to get started with development in a realistic environment that then scales well to production. It works in the way that it hosts a grafana instance and a influxdb instance that then talk to each other so grafana can query the data and form diagram from that.

You can of course use pybytes which offers an awesome integration with the lopy4 it will be much easier to setup a simple dashboard there and get it to work, but to go from there to a production setup is a much bigger step than from docker instance since its more realistic setup for an IoT project. But in this we go with ubuntu and docker.

A little guide below how to setup similar instance.

If you don't have docker then go ahead and download it from here: https://www.docker.com/get-started

Below image was used.
https://github.com/samuelebistoletti/docker-statsd-influxdb-grafana

Run below command to pull the latest image the run the next command to download and install

# docker pull samuelebistoletti/docker-statsd-influxdb-grafana
# docker run --ulimit nofile=66000:66000 \
  -d \
  --name docker-statsd-influxdb-grafana \
  -p 3003:3003 \
  -p 3004:8888 \
  -p 8086:8086 \
  -p 8125:8125/udp \
  samuelebistoletti/docker-statsd-influxdb-grafana:latest

Since database cant be changed in the webUI due to everything being built by configuration files, if you want to change the default datastore follow below instructions.

Change the uppercase letters below to your database credentials. Open the file with `nano` to edit.

# cd /etc/grafana/provisioning/datasources
# cat influxDB.yml
datasources:
  - name: InfluxDB
    type: influxdb
    access: proxy
    database: DATABASE
    user: USER
    password: PASS
    url: http://localhost:8086
    isDefault: true

## The code

#### boot.py
https://docs.pycom.io/tutorials/all/wlan

#imports machine modules
import machine 
#import wlan from network module
from network import WLAN

#sets the wlan mode to a station
wlan = WLAN(mode=WLAN.STA)
#scans for available network
nets = wlan.scan()
#for each net check if SSID checksout with the one given
for net in nets:
    if net.ssid == 'SSID':
        print('Network found!')
        #connect with the wpa key
        wlan.connect(net.ssid, auth=(net.sec, 'WPA-2 KEY'), timeout=5000)
        while not wlan.isconnected(): # otherwise idle if net is not found
            machine.idle() # save power while waiting
        print('WLAN connection succeeded!') # notify connection success
        break

#### main.py
https://github.com/Hortlund/1DT305-Applied-IoT/blob/master/main.py

#Import all pycom and micropython modules needed 
import pycom
import machine
import _thread
import time
import ujson as json

#This import the urequests folder as requests with the urequests.py file inside
#This is needed because of the way the filesystem is structured on the pycom device
import urequests as requests


adc = machine.ADC() # creates a analog to digital conversion object
apin = adc.channel(pin='P16') # maps pin p16 to apin variable
bpin = adc.channel(pin='P15') # maps pin p15 to bpin variable

url = 'URL' #url for upload to database


while(1): # infinite loop
    temprature = (bpin.voltage() - 500) / 10 # calculation to get temprature from tmp 36gz thermistor
    lux = (250/(0.004*apin.voltage())-50) # rough calculation to get lux value from ldr, give a somewhat correct response
    print("Temprature: {:.1f}".format(temprature)) #prints temp
    print("Lux: {:.1f}".format(lux)) #prints lux
    #structures the payload so that influxdb can direclty receive it, with temp and lyx as part of the body
    payload = "sensors,value=\"Temp\" temprature={:.1f}\nsensors,value=\"Light\" lux={:.1f}".format(temprature,lux)
    requests.post(url,data=payload) # sends the data using urequests.py function
    print("data sent...")
    time.sleep(10) # waits 10 sec before doing it again

#### urequests.py
https://github.com/micropython/micropython-lib/blob/master/urequests/urequests.py

import usocket

class Response:

    def __init__(self, f):
        self.raw = f
        self.encoding = "utf-8"
        self._cached = None

    def close(self):
        if self.raw:
            self.raw.close()
            self.raw = None
        self._cached = None

    @property
    def content(self):
        if self._cached is None:
            try:
                self._cached = self.raw.read()
            finally:
                self.raw.close()
                self.raw = None
        return self._cached

    @property
    def text(self):
        return str(self.content, self.encoding)

    def json(self):
        import ujson
        return ujson.loads(self.content)


def request(method, url, data=None, json=None, headers={}, stream=None):
    try:
        proto, dummy, host, path = url.split("/", 3)
    except ValueError:
        proto, dummy, host = url.split("/", 2)
        path = ""
    if proto == "http:":
        port = 80
    elif proto == "https:":
        import ussl
        port = 443
    else:
        raise ValueError("Unsupported protocol: " + proto)

    if ":" in host:
        host, port = host.split(":", 1)
        port = int(port)

    ai = usocket.getaddrinfo(host, port, 0, usocket.SOCK_STREAM)
    ai = ai[0]

    s = usocket.socket(ai[0], ai[1], ai[2])
    try:
        s.connect(ai[-1])
        if proto == "https:":
            s = ussl.wrap_socket(s, server_hostname=host)
        s.write(b"%s /%s HTTP/1.0\r\n" % (method, path))
        if not "Host" in headers:
            s.write(b"Host: %s\r\n" % host)
        # Iterate over keys to avoid tuple alloc
        for k in headers:
            s.write(k)
            s.write(b": ")
            s.write(headers[k])
            s.write(b"\r\n")
        if json is not None:
            assert data is None
            import ujson
            data = ujson.dumps(json)
            s.write(b"Content-Type: application/json\r\n")
        if data:
            s.write(b"Content-Length: %d\r\n" % len(data))
        s.write(b"\r\n")
        if data:
            s.write(data)

        l = s.readline()
        #print(l)
        l = l.split(None, 2)
        status = int(l[1])
        reason = ""
        if len(l) > 2:
            reason = l[2].rstrip()
        while True:
            l = s.readline()
            if not l or l == b"\r\n":
                break
            #print(l)
            if l.startswith(b"Transfer-Encoding:"):
                if b"chunked" in l:
                    raise ValueError("Unsupported " + l)
            elif l.startswith(b"Location:") and not 200 <= status <= 299:
                raise NotImplementedError("Redirects not yet supported")
    except OSError:
        s.close()
        raise

    resp = Response(s)
    resp.status_code = status
    resp.reason = reason
    return resp


def head(url, **kw):
    return request("HEAD", url, **kw)

def get(url, **kw):
    return request("GET", url, **kw)

def post(url, **kw):
    return request("POST", url, **kw)

def put(url, **kw):
    return request("PUT", url, **kw)

def patch(url, **kw):
    return request("PATCH", url, **kw)

def delete(url, **kw):
    return request("DELETE", url, **kw)

## Transmitting data & Connectivity
The choice of protocols for this fell quite naturally, since the lopy will be stationary with power always plugged in and in range of a Wi-Fi connection, I choose to go with the Wi-Fi protocol for connection, since i wont be bothered with connectivity issues due to range or bad bandwidth that can happen with sigfox or lora connection.

The data is sent every 10 seconds via an http post request, the body holds the data of the sensors and to what measurement it should be stored in at the database. I choose the http protocol for the data because of its ease of use and that's is a really mature standard. I also have no limit of the size of the data sent, although that's not going to be a problem since it only 2 float values that's being sent.

## Presenting the data
*The spike seen here is when a flashlight was pointed towards the LDR.*


Why I choose influxdb is because its a time series database which means it designed to structure data against time points so it makes it ideal for diagrams measuring data over time.

The data is saved every time it is being received which is every 10th second and the database is set to a retention period of forever, so the data is stored there until you manually delete it. This is to be able to see the effects of the data under a long period of time, and since storage is not an issue i could do that, if storage is limited then changing the retention period would help.

## Finalizing the design
Below are some images from the project assembled. Not very unique or interesting in anyway but a finished project nonetheless. I think the project went well, I didn't face any bigger issues because the project is of a very basic sort. The one thing that took the most time was for me to figure out how influxdb took parameters and stored them.

Something that could have been done better is the payload structure, I never got influx to accept json as data, if one could fix that it would be a more clear syntax of the payload and easier to structure for influxdb also.

Very much an interesting course and looking forward for a more advanced one.