### Oskar Almqvist - oa222my
###### oa222my

# (VibSense) - How to build yourself a third hand by creating a IOT assistant capable of sensing vibrations and impacts with ultra high sensitivity.

:::warning
This project showcases how to build an IOT device/assistant capable of combining the output of multiple vibration sensors to achieve a "ultra sensitive" way to detect impacts, shocks and vibrations using two simple, inexpensive, commodity sensors. Data is stored in a .JSON database for further analysis and sent to a dashboard for "live monitoring"
:::

**Project cost:** 100 $
**Time to replicate:** Using this tutorial and the code from the Github repo it should be possible to replicate this project in 2-4 hours depending on prior experience and background.

---

## Objective


I've been toying with the idea of moving out the country for years and spent quite a few years now looking at prospects. Many of these places completely lack the things we in cities take for granted. Such as running water, electricty and of course internet connectivity.

The good news is that while the lack of these amendenties historically has been a great pain in the ass, technology - has caught up.

Electricity? Solar panels. Water? Purification or drilling. Internet? 4G, LoRa, SigFox and satellites!

### However.. One problem remain!
**There is no people!!**

So how do we solve issues like theft or watching over a dog? How do make sure a tree or a storm isn't causing our entire house to tremble?


# We build ourselves a third hand!
VibSense is just that. A third hand! By combining the power of multiple, cheap vibration sensors with internet connectivity we create a device capable of assisting us in all kinds of scenarios.

* Worried about theft?
* Place VibSense in your boat or car and get an alert when something is happening
* Worried about your dog?
* Place VibSense in your dogs resting place to get frequent updates about his movements
* When the heck is the mailman arriving?
* You guessed it, place VibSense inside your mailbox!

VibSense senses duration of vibrations as well as the impact of shocks and can be calibrated for all kinds of uses. It's basically your third hand!

---

## Material


At first I purchased only the low sensitivity vibration sensor (41015724) and my plan was to combine that sensor with some other type of sensor for the hand. However, as I tested out that sensor I realized that while it's somewhat capable of detecting the actual impact of the shock it's not very capable at detecting the duration of the shock.

The problem with this is that while a "dangerous vibration" for a procelain vase obviously is the actual force of the impact the "dangerous vibration" for a bowl of fish is something difference. Minor shakes but where the shaking continue for longer periods of time might be just as dangerous, if not more dangerous to a living thing even if the actual force of impact is small. Since I wanted a device capable of being useful in both scenarios I decided to combine this sensor with the high sensitivity vibration sensor 41015741.

| Item | Bought At | Price € |
|:---------------------------------- |:-------------- | -------:|
| LoPy4 | pycom.io | 39 |
| Pycom Expansion Board 3.0 | pycom.io | 18 |
| Vibration sensor (low) - 41015724 | electrokit.com | 3 |
| Vibration sensor (high) (41015741) | electrokit.com | 4 |
| Portable battery pack | electrokit.com | 3 |
| Breadboard | electrokit.com | ? |
| Wires | electrokit.com | ? |
| Micro USB cable | electrokit.com | ? |
| | | 67 € |

# Material

**Fig 1. LoPy4** A small development board capable of being programmed with MicroPython. This board is able to connect to various networks such as WiFi, Blueooth, LoRa and Sigfox.

**Fig 2. Expansion Board** Provides an easy way to connect things like sensors and battery to the LoPy4. Also has an USB-port which makes interfacing with, and powering the LoPy4 significantly easier.

**Fig 3. Portable battery pack** Enables the device to run without being connected to an outlet. Easy to charge and most people have them available for free at home.

**Fig 4. Vibration Shake Sensor 41015724** The switch primarily consists of a terminal that forms a center post and a second terminal that is a spring that surrounds the center post. When a sufficient force is transferred to the switch, the terminal consisting of the spring moves and shorts both terminals together.

**Fig 5. High sensitivity Vibration Shake Sensor 41015741** The switch primarily consists of a terminal that forms a center post and a second terminal that is a spring that surrounds the center post. When a sufficient force is transferred to the switch, the terminal consisting of the spring moves and shorts both terminals together.
* This sensor has significantly higher sensitivity than 41015724

**Fig 6. Wires and breadboard**
Wires and breadboards are used to connect the sensors to the LoPy. It is possible to make the project without the breadboard.


---

## Computer & hardware setup

To communicate with the devices you need to install some software to your computer.

In this tutorial I've used VSCode for writing all the code. But this can also be done using the Atom IDE.

If you're using a new Pycom device to execute this tutorial than it's highly recommended that you update the firmware of the expansion board and the device itself to get access to all new features and make sure that any bugs are corrected.

There is very good documentation about how to achieve this on the Pycom website accessible here:

https://docs.pycom.io/updatefirmware/

To install the IDE used in this tutorial you can download Visual Studio Code here:

https://code.visualstudio.com/docs/setup/windows

In addition you need the PyMakr plugin which can be installed here:

https://marketplace.visualstudio.com/items?itemName=pycom.Pymakr
PyMakr allows us to communicate with the LoPy and is used to run code on the board and to upload files to the board.

---

## Putting everything together


**Breadboard** the breadboard is connected with black to GND on the LoPy and - on the BreadBoard. It is also connected with red to 3V3 on the LoPy and + on the breadboard to power the sensors.

**Vibration Shake Sensor 41015724** is attached to the breadboard and connected to power and ground via the breadboard and to PIN16 on the LoPy via the breaboard.

:::info
The reason the vibration shake sensor is not attached directly to the breadboard is that after testing it for a hundred+ times I saw that the readings got significantly worse when directly attached to the breadboard. Probably because the breadboard itself then absorbs some of the vibrations. The high sensitivity sensor does not have this problem.
:::

**High sensitivity Vibration Shake Sensor 41015741** is not attached to the breadboard but connected to power and ground via the breadboard and to PIN13 on the LoPy via the breadboard.

:::danger
The official image is WRONG and the sensor has been reworked. There is now three pins. From up to down - VCC (power), GND (ground) and DO (signal).
:::

**Portable battery pack** is connected to the MicroUSB-port on the expansion board to provide power.

**Case** To make a simple case I used an old lunchbox wrapped with bubblewrap to hold all the components together. This could actually be somewhat waterproof but there is a drawback in that the components aren't properly fixed and could fall out in case of a extreme shock causing incorrect readings. This can be improved upon.
:::info
The case has been wrapped in bubblewrap to protect the LoPy from vibrations. This is recommended :)
:::

### Battery calculations

According to the Chinese manufacturer the USB power bank has a capacity of 13000 mAh. Once testing the device we will see that this actually isn't the case. The power bank is slightly dated and in addition the batteries inside probably isn't of the highest quality.

For the sake of this tutorial we're going to calculate what would happen if we actually got 13000 mAh out of the power bank.

Looking at previous IOT projects posted and through the Pycom forums it's estimated that the device draws about 200 mA while awake transmitting through WiFi with the expansion board connected and multiple sensors.

To calculate the battery life we can make a very simple calculation

Battery life in hours = (battery capacity mAh)/current in mA

If we then use a 13000 mAh power bank and the average current is 200 ma we get

x/y = h, 13000/200 = 65 = 65 hours of run time for the portable power bank.

:::warning
This can be extended by playing around with deep sleep and other connectivity methods that draws less power.
:::

:::danger
These calculations is a rought estimation and might deviate in actual usage.
:::

---

## Platform


PyBytes is the cloud-based device management platform offered for free for all PyCom devices. PyBytes gives us the ability to track our device (is it online, etc) and to visualize data in dashboards. It has a ton of other features except for this like webhooks, firmware upgrades etc.

I know that previous students of this course decided against using [Pybytes](https://pybytes.pycom.io/) mainly because the platform only retains your data for 30 days. However, that is plenty for this project as the IOT device we're building provides instant feedback and historical data is of less importance.

As I am familiar with web development getting a webhook from PyBytes to my server and then coding that data myself in a web language to suit my needs is preferable to setting everything up on a more complicated platform.

In addition, all the data collected by the device is saved to a local .JSON file on the device which means that we can at any point in time retreive the device, download the data and perform any kind of analysis on the data.

---

## The code

[The full code is published on GitHub](https://github.com/MrSvenson/IOTLinneusKurs).

I've separated the code into several files and included heavy commentary in the code to make it easier to follow. The algorithm is contained in algorithm.py and can be modified freely depending on how you're using the device.

In addition I've used try/except blocks where needed to make the device less sensitive to unexpeected errors.

### boot[]().py
Here we initialize everything and make sure to disable bluetooth to save power.

import pycom
from network import Bluetooth, LTE
from machine import UART
import machine
import os

uart = UART(0, baudrate=115200)
os.dupterm(uart)

# Disable bluetooth to save power
bluetooth = Bluetooth()
bluetooth.deinit()

machine.main('main.py')

### main[]().py
Magic. Here we get data from all the sensors, compare it and send it to PyBytes if it matches our criteria.

## Data is sent to Pybytes. Needs to flashed with Pybyte firmware
from machine import Pin
from machine import ADC
import time  # import library called time
import pycom  # import library called pycom
import json # import json
import ssl # import ssl
import functions # import our functions file
import sensors # import sensors file
import algorithm # import the algorithm

# Making sure the "connection" to pybytes is activated
pycom.pybytes_on_boot(True)

# Set time using NTP server
rtc = functions.set_time()

VibSensorPin = 'P16' # Knock sensor connected to P16. Valid pins are P13 to P20.
VibSensorPinHigh = 'P13'  # Vibration sensor connected to P13. Valid pins are P13 to P20.

Pin(VibSensorPin, mode=Pin.IN)  # set up pin mode to input
Pin(VibSensorPinHigh, mode=Pin.IN)  # set up pin mode to input

# create an ADC object bits=10 means range 0-705 the lower value the more vibration detected
adc = ADC(bits=10)

# create analog pins on P16 and P13;
# attn=ADC.ATTN_11DB measures voltage from 0.1 to 3.3v
apin = adc.channel(attn=ADC.ATTN_11DB, pin=VibSensorPin) # Low sensitivity sensor
apin_high = adc.channel(attn=ADC.ATTN_11DB, pin=VibSensorPinHigh) # High sensitivity sensor


if __name__ == '__main__':
    # Loop forever
    while True:
        try:
            # Do one initial reading. If we don't detect anything here then we continue waiting for something to happen
            low_sensitivity_sensor = apin()   # analog value, low sensitivity sensor
            high_sensitivity_sensor = apin_high() # analog value, high sensitivity sensor

            # Everything above 600 is too little to care about. Ignoring that.
            # Everything below 20 (high sensitivty) is also too little/nothing. We ignore that as well.
            if low_sensitivity_sensor < 650 or high_sensitivity_sensor > 20:
                print("Detected a shock! Running application")

                #  Get data from the high sensititivty sensor
                data = sensors.detect_duration_of_shake(apin_high)
                high_sensitivity_duration = data[0] # Duration in MS of the vibration
                shake_readings = data[1] # How many times during the function did we get a value above 0 indicating vibration is occuring.

                # Get data from the low sensitivity sensor
                low_sensitivity_sensor = sensors.detect_shock_value(apin)

                # Get a calibrated value using the two sensors
                calibrated_data = algorithm.vibration_formula(
                    low_sensitivity_sensor, shake_readings)

                # Store data, read data. For testing mainly
                functions.store_file_json_new(
                    low_sensitivity_sensor, high_sensitivity_duration, calibrated_data['calibrated_value'])
                # functions.read_json()


                if calibrated_data['calibrated_value'] > 600:
                    pretty_status = "SICK"
                    signal = 1
                elif calibrated_data['calibrated_value'] > 500:
                    pretty_status = "BIG"
                    signal = 2
                elif calibrated_data['calibrated_value'] > 300:
                    pretty_status = "Medium"
                    signal = 3
                else:
                    pretty_status = "Light"
                    signal = 4

                # Console printing
                print("-------------------------", pretty_status," shock!-------------------------")
                print("Raw value (knock): ", low_sensitivity_sensor)
                print("Duration of shake: ",high_sensitivity_duration)
                print("Low sensor score: ",calibrated_data['low_sensor_score'])
                print("High sensor score: ",calibrated_data['high_sensor_score'])
                print("Calibrated score: ",calibrated_data['calibrated_score'])

                #C = Calibrated value, using the formula above and combining the readings of the two sensors. This is defined in algorithm.py
                #C = Right now more emphasis is put on the low sensivity sensors which mean that we increase how much we care about the shock of the impact and less about the impact duration
                #R = Raw data from the knock sensor, this detects how hard the impact is. Did we drop it from 10 cm height or from 2 meter height?
                #D = Duration from the vibration sensor, for how long did the impact last? Did it rattle around?

                # Pybytes send
                pybytes.send_signal(
                    signal, 
                    ""+pretty_status+": C: "+str(sensors_calibrated_value)+"/R: "+str(low_sensitivity_sensor)+"/D: "+str(round(high_sensitivity_duration))+"ms")

                #  Sleep after a impact to give the machine time to reset.
                time.sleep(1)

        except Exception as e:
            print(str(e))

### algorithm[]().py
The formula used in this file has been calibrated by basically throwing stuff around in my apartment :)

Here is some math behind it:
### Low sensitivty sensor
Low sensitivity sensor gives 720 when it hasn't detected anything.
The score then for that in idle is 720-720 = 0

### High sensitivty sensor
The high sensitivty sensor gives 0 when it hasn't detected anything.
If it detects something then it calculates for how long the shake is happening.
This is recorded in milliseconds. The shake_duration parameter is 100 milliseconds.
I.E shake_duration = 4 = 400ms.

# Math
# A small impact would give us
# 720-600 = 120 (low sensor)
# 1 (100ms)*200 = 200 (high sensor)
# A small impact == 300

# A big impact?
# 720-200 = 520 (low sensor)
# 2*200 = 400 (high sensor)
# A really big impact would therefore be 520+400 = 920.

def vibration_formula(low_sensitivity_sensor, shake_duration):
	# The knock sensor returns a low_sensitivity_sensorue from 700 (nothing detected), down to 0 (big impact detected). This is less accurate
	low_sensor_score = 720-int(low_sensitivity_sensor)
	low_sensor_score = low_sensor_score*4  # Increase importance of knock by a factor of 4.

	# The vibration sensor returns a low_sensitivity_sensor from 0 (nothing detected) up to 1024 (vibration detected)
	# By taking the vibration time into account, we can consider how extreme the impact is.
	# This is more accurate sensor, so we place more emphasis on this low_sensitivity_sensor
	# So if we detected a shake for 4'isch second this would give us 4*100 = 400
	high_sensor_score = shake_duration*100

	calibrated_score = int(round(low_sensor_score)+int(high_sensor_score))

	data = {
		"low_sensor_score": low_sensor_score,
		"high_sensor_score": high_sensor_score,
		"calibrated_score": calibrated_score
	}
	return data

In main.py we use the data like this. Of course both this and the calculation in algorithm.py can be changed depending on if you want to put more emphasis on the force of the impact or the duration of the shake.
I calibrated this by throwing stuff around in my apartment and so far this is the best calibration values I could come up with.

if calibrated_data['calibrated_score'] > 600:
    pretty_status = "SICK"
    signal = 1
elif calibrated_data['calibrated_score'] > 500:
    pretty_status = "BIG"
    signal = 2
elif calibrated_data['calibrated_score'] > 300:
    pretty_status = "Medium"
    signal = 3
else:
    pretty_status = "Light"
    signal = 4

### sensors[]().py

import functions
import time
from machine import Timer
	
def detect_duration_of_shake(apin_high):
    # Small shake = 800ms
    # Two small shakes = 1200ms
    # Three smalle shakes = 1600ms
    try:
        # print("Detecting shake duration")
        chrono = Timer.Chrono()
        chrono.start() # Start a timer to record how long it's been shaking for.
        shake_readings = 0
        while True:
            value = 0
            # Read the data 400 times over 1 second to detect if it stopped shaking or not.
            for i in range(400):
                value = value+apin_high()
                time.sleep(0.001)
            if value == 0: # If we don't detect any shaking over 1 second then we assume shaking stopped and break out of loop
                break
            else:
                shake_readings += 1 # Otherwise continue reading the sensor.
        chrono.stop()
        total_ms = chrono.read()*1000
        return [total_ms, shake_readings] #Return the execution time of the function in ms, and the shake readings 1 == 100ms.
    except Exception as e:
        print(str(e))


# Read the shock_value 100 times and get an average reading. 
# We do this in case there is any problems with the readings due to interference from the high sensitivity sensor.
def detect_shock_value(apin):
    value = 0
    for i in range(100):
        value = value+apin()
        time.sleep(0.001)
    return int(value)/100

### functions[]().py
Many functions for storing/reading data, setting RTC-time etc.
The main function in this file is the ability to store the data we collect in a .JSON file. This is great for analysis/investigation at a later point without having to send all the data to the cloud.

from machine import RTC
import time  # import library called time
import json

# Store data in a .JSON file.
# This is great since we later have the option to analyze and use all the data for calculations.

def store_file_json_new(low_sensitivity_sensor, high_sensitivity_duration, cal):
    try:
        FILENAME = 'data.json'
        new = False
        try:
            f = open(FILENAME, "r")
            HAS_FILE = True
            f.close()
        except:  # open failed
            HAS_FILE = False
        if not HAS_FILE:
            new = True
            my_list = []
        else:
            with open(FILENAME, 'r+') as file:
                my_list = json.loads(file.read())
        with open(FILENAME, 'w+') as file:
            entry = {
                'pretty_status': pretty_status, # Pretty status
                'low_sensitivity_value': low_sensitivity_sensor, # Raw value from the low sensor 
                'high_sensitivity_duration': high_sensitivity_duration, # Raw duration value
                'shake_readings': shake_readings, # Same as duration, 1 == 100ms
                'low_sensor_score': calibrated_data['low_sensor_score'], # Calibrated score of the low sensor
                'high_sensor_score': calibrated_data['high_sensor_score'], # Calibrated score of the high sensor
                'calibrated_score': calibrated_data['calibrated_score'], # Low+high sensor calibrated together.
                'time': str(time.localtime())
            }
            my_list.append(entry)
            jsonString = json.dumps(my_list)
            file.write(jsonString)
    except Exception as e:
        print(e)

---

## Transmitting the data / connectivity


[WiFi](https://en.wikipedia.org/wiki/Wi-Fi) is a family of wireless network protocols, based on the IEEE 802.11 family of standards, which are commonly used for local area networking of devices and Internet access, allowing nearby digital devices to exchange data by radio waves.

This project transfers data to Pybytes using WiFi. A 4G bridge (mobile hotspot) is adviced, there is of course also the opportunity to easily implement LoRa.

The problem with WiFi is the lack of range and the increased battery usage. However, the range is easily corrected by using a 3G/4G bridge (cellular hotspot).

Data is transferred to Pybytes via the MQTT protocol every time a impact is detected by the device.

---

## Presenting the data


Pybytes is a good way to get an overview of the VibSense. Despite being a simple platform we're able to get a ton of useful data displayed.

First we have a pretty printed status (SICK, BIG, Medium Light) giving us an easy overview of how severe the latest impact was, as well as when it happened.

We can also see the calibrated value (C) (the combined value of both sensors from the algorithm), the raw value (R) from the low sensitivity sensor which represents the force of the last impact (the force of the impact) as well as the duration of the impact (D) from the high sensitivity sensor.

Of course, we also have all the data available in a .JSON file on the device which can be downloaded at any time for further analysis.

[
   {
      "pretty_status":"Light",
      "calibrated_score":140,
      "high_sensitivity_duration":800.6931,
      "time":"(2021, 7, 22, 10, 21, 25, 3, 203)",
      "shake_readings":1,
      "high_sensor_score":100,
      "low_sensitivity_value":710.69,
      "low_sensor_score":40
   },
   {
      "pretty_status":"Medium",
      "high_sensitivity_duration":1200.734,
      "shake_readings":2,
      "calibrated_score":352,
      "time":"(2021, 7, 22, 10, 21, 37, 3, 203)",
      "high_sensor_score":200,
      "low_sensitivity_value":682.99,
      "low_sensor_score":152
   },
   {
      "pretty_status":"BIG",
      "high_sensitivity_duration":800.9038,
      "shake_readings":1,
      "low_sensor_score":504,
      "calibrated_score":604,
      "high_sensor_score":100,
      "time":"(2021, 7, 22, 10, 21, 41, 3, 203)",
      "low_sensitivity_value":594.08
   }
]

### Webhook and notifications
PyBytes allows us to [set a webook](https://docs.pycom.io/pybytes/integrations/webhooks/) to forward all incoming data to another server.

I decided to integrate this with the application [Pushover](/pushover.net/apps/a9uc6ybiy925xfaz5wyxwkpzgrvynb) which is a smartphone app that enables you to send pushnotifications to your phone using WEB-requests.

To do that I created a new application in the Pushover dashboard


I then made a simple PHP-script and placed it on a webserver

<?php
curl_setopt_array($ch = curl_init(), array(
  CURLOPT_URL => "https://api.pushover.net/1/messages.json",
  CURLOPT_POSTFIELDS => array(
    "token" => "XXXX",
    "user" => "YYYY",
	"title" => "Shock detected!",
    "message" => $_POST['payload']
  ),
  CURLOPT_SAFE_UPLOAD => true,
  CURLOPT_RETURNTRANSFER => true,
));
curl_exec($ch);
curl_close($ch);
?>

To summarize this architecture
* VibSense sends data to PyBytes
* PyBytes receives the data, sends it to a PHP-script on a webserver
* The PHP-script takes the data, sends it to Pushover
* We get a push notification on our phones

## Pushover result

It doesn't get more awesome than that!

---
## Insights

I think the most important insights i've gained throughout this project is:
* How cheap IOT sensors is. For a dollar or two you can get a vibration, temperature or light sensor that is quite capable.
* How big of a difference it makes to take TWO sensors, combine the input/output and make conclusions from that.

Running the project with only one sensor connected highlights the difference. We get significantly worse readings and suddenly the device is prone to all kinds of errors. For example the side of the box where the sensor is located might be hit extra hard, or extra soft giving us a inaccurate reading of the force of the impact. This is almost completely eliminated by using dual sensors.

---

## Final product

{%youtube LALFXFXRjiQ %}

There is room for improvements. LoRa can be implemented to support long-range activities without a cellular bridge. The code can be cleaned up and explained better. You could also include even more vibration sensors to get a even more accurate reading.