#### 1DT305 Tillämpad IoT - Tutorial - Lexx Damm, adaha09

## Human agent biometric and stat supervisor system

Lexx Damm, adaha09

My aim with this project is to prototype a future system for personel biometric and stats supervisor system for my own fictional group. The system should comprise of a system of wearable devices that gather various biometric, enviromental and position metrics. I estimate it would take 1 to 3 hours to wire up, write the code and configure the systems.

### Objective

I decided early on that I wanted to do something fun and futuristic. I am into cybernetics and wearables and one of my favorite sci-fi movie is Aliens 2 which if I remember correctly have a supervisor system for the marines which show among other things camera, ECG and some other things. I also wanted to incoperate my own NFC-implants into the design somehow by making my wearable(s) NFC-connected.

I believe my prototype will make it more clear the pratical use of such a supervisor system and what drawbacks or unforseen problems that may present itself. I also believe it will prove a practical use of my implants.

In my project I will only focus on a monitor system that is supervised, basically a human being or AI monitoring personel stats in real-time.

### Material

List of materials:
| Item | Cost | Supplier |
| ------------- | - |-|
| Pycom Lopy4 | 34.95 EUR| Pycom webshop|
| Pycom Fipy | 54 EUR| Pycom webshop|
| Pycom Pyscan | 39.90 EUR | Pycom webshop|
| Pycom Pytrack | 34.95 | Pycom webshop|
| Playknowlogy Komponent-kit | 299.90 SEK | Kjell & co|
| Mikroe ECG 4 CLICK kit | 699.09 SEK + TAX | RS Components |

In my project the main processing devices I have chosen to use is the Pycom line of developer boards. I chose them because it was the recommended device to use, but personally I like the balance between down to the metal but also easy of use it provides but at a good performance and minimal physical space needed.

I decided early on that I wanted to do something with both NFC and GPS, so I ordered the Pyscan and Pytrack extension boards for both my Pycom.
I looked up if there was a ECG board available and found the Click line of boards from Mikroe. I first bought the ECG 2 CLICK but after some testing realized it was to complicated so I ordered the more simpler ECG 4 CLICK from Mikroe which use a UART interface and simple communication and data protocol.

### Computer setup

Software links:
- Atom
https://atom.io/
- pycom-fwtool ubuntu deb
https://pycom.io/downloads/

I used the Atom IDE on Ubuntu 20.04 and installed the pymakr package through the Atom package manager. I also installed pycom-fwtool to update my Pycom boards to the latest firmware. I uploaded the code to the two Pycom devices via Pymakr.

### Putting everything together

The FiPy device is plugged into the Pytrack extension board while the LoPy is plugged into the Pyscan board. I then used a breadboard to plug in the Mikroe ECG 4 CLICK and wired the UART lines (TX and RX) to one of the headers on the Pyscan board.




I didn't have the time or resources to make a custom PCB, a housing for the boards so my setup is only for prototyping.

### Platform

I used the The Things Network to gather my data from LoRaWAN and then forwarded it to Ubidots to visualize the data.

For my prototype I used a simple dashboard on Ubidots visualizing the GPS data with a map and the ECG data with a value for the interval between heartbeats.

### The code

comms - FiPy code:

import machine
import time
import pycom
import socket
import ubinascii
import struct
import _thread
from network import LoRa
from L76GNSS import L76GNSS
from pytrack import Pytrack

# TTN EUI and APP KEY
app_eui = ubinascii.unhexlify('70B3D57ED0031632')
app_key = ubinascii.unhexlify('10D4E433364CF8A3E78F0708BA22EC42')

pycom.heartbeat(False)
pycom.rgbled(0x060000)
lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868)
py = Pytrack()
l76 = L76GNSS(py, timeout=30)

lora.join(activation=LoRa.OTAA, auth=(app_eui, app_key), timeout=0)
pycom.rgbled(0x0a0a00)

while not lora.has_joined():
    print('Not yet joined')
    time.sleep(3)

print('Joined LoRa TTN')
pycom.rgbled(0x000f00)

sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
sock.setsockopt(socket.SOL_LORA, socket.SO_DR, 5)
sock.setblocking(True)
sock.bind(1)

def gps_loop():
    while (True):
        coord = l76.coordinates()
        if (coord[0] == None and coord[1] == None):
            pycom.rgbled(0x0a0a00)
            continue
        pycom.rgbled(0x00000a)
        print('Coordinates: {:.6f}, {:.6f}'.format(coord[0], coord[1]))
        pack = bytearray(struct.pack('f', float(coord[0])) + struct.pack('f', float(coord[1])))
        sock.send(pack)
        time.sleep(60 + 60 + 60 + 60 + 60)

_thread.start_new_thread(gps_loop, ())

id - LoPy4 code:

iimport time
import pycom
import _thread
import ubinascii
import math
from machine import UART
from machine import Pin
from machine import Timer
from pyscan import Pyscan
from MFRC630 import MFRC630
from network import LoRa

# TTN EUI and APP KEY
app_eui = ubinascii.unhexlify('70B3D54993281B0B')
app_key = ubinascii.unhexlify('3987D8D92D72242591EC37D7E327A7F8')

# Dictonary of stored uid and their user is
my_ids_dict = {
    b'\x04~e\x12\xff8\x81\x00\x00\x00': b'\x01',
    b'\x04\x19\x08\x12\xff8\x85\x00\x00\x00': b'\x02'
}

# 2's compliment to get 16 bit value from 2 bytes
def twos_comp(val, bits):
    if (val & (1 << (bits - 1))) != 0:
        val = val - (1 << bits)
    return val

# Wait for scanning the correct NFC implant
def auth_thread_loop():
    global auth_busy
    global auth_logged_in
    global auth_user_id
    auth_busy = True
    auth = False
    while (not auth):
        atqa = nfc.mfrc630_iso14443a_WUPA_REQA(nfc.MFRC630_ISO14443_CMD_REQA)
        if (atqa != 0):
            uid = bytearray(10)
            uid_len = nfc.mfrc630_iso14443a_select(uid)
            if (uid_len > 0):
                print('UID: {}'.format(uid))
                for stored_uid, user_id in my_ids_dict.items():
                    if (uid == stored_uid):
                        print('Logged in')
                        auth_logged_in = True
                        pycom.rgbled(0x000f00)
                        auth_user_id = user_id
                        auth = True
                        auth_busy = False
                        break

# ECG loop
def ecg_thread_loop():
    global ecg_busy
    global ecg_interval
    ecg_busy = True
    ecg_interval = None
    ecg_state = 'PRE'
    ecg_data = []
    counter = 0
    while (counter < 2000):
        if (uart.any()):
            char = uart.read(1)
            if (uart.any() and char and char[0] == 0xaa):
                char = uart.read(1)
                if (uart.any() and char and char[0] == 0xaa):
                    char = uart.read(1)
                    if (uart.any() and char and char[0] == 4):
                        payload_len = char[0]
                        payload = uart.read(payload_len)
                        if (payload):
                            crc = uart.read(1)
                            code = payload[0]
                            if (code == 0x80):
                                data_row_len = payload[1]
                                data_row = bytearray(payload[2:4])
                                ecg_value = twos_comp(int(ubinascii.hexlify(data_row), 16), 16)
                                ecg_data.append(ecg_value)
                                counter += 1

    i = 0
    found = False
    for value in ecg_data:
        print(value)
        if (ecg_state == 'PRE' and value > 1800):
            ecg_state = 'PEAK'
        if (ecg_state == 'INTER' and value > 1800):
            ecg_state = 'PEAK'
        if (ecg_state == 'PEAK' and value < -1800):
            ecg_state = 'DIP'
        if (ecg_state == 'DIP' and value < 200 and value > -200):
            ecg_state = 'INTER'
            if (found == True):
                print('Found interval')
                ecg_interval = i
                ecg_busy = False
                return ecg_interval
            else:
                i = 0
                found = True
        i += 1
    ecg_busy = False

def send_lora_bytes(port, bytes):
    sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
    sock.bind(port)
    sock.setblocking(True)
    sock.send(bytes)

pycom.heartbeat(False)

auth_uplink_timer = Timer.Chrono()
ecg_uplink_timer = Timer.Chrono()

# Init the UART and configure it to use the pins 19 and 15 as UART TX and RX pins connected to the Mikroe ECG 4 CLICK uart
uart = UART(1, 57600, bits=8, parity=None, stop=1, pins=('P19', 'P15'))

py = Pyscan()
nfc = MFRC630(py)
lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868)

nfc.mfrc630_cmd_init()

pycom.rgbled(0x0f0000)

lora.join(activation=LoRa.OTAA, auth=(app_eui, app_key), timeout=0)

auth_busy = False
auth_logged_in = False
auth_user_id = None
ecg_busy = False
ecg_interval = None
auth_uplink_timer.start()
ecg_uplink_timer.start()
while (True):
    print('Waiting for LoRa connection to TTN')
    if (lora.has_joined()):
        print('Connected to TTN via LoRa')
        if (auth_uplink_timer.read() > 300):
            if (not auth_busy):
                _thread.start_new_thread(auth_thread_loop, ())
            if (auth_logged_in):
                print('Sending user login')
                bytes_pack = bytearray(struct.pack('c', auth_user_idl))
                send_lora_bytes(2, bytes_pack)
                auth_uplink_timer.reset()
                auth_logged_in = False
        if (ecg_uplink_timer.read() > 300):
            if (not ecg_busy):
                _thread.start_new_thread(ecg_thread_loop, ())
            if (ecg_interval != None):
                print('Sending ecg interval')
                bytes_pack = bytearray(struct.pack('f', ecg_interval))
                send_lora_bytes(3, bytes_pack)
                ecg_uplink_timer.reset()
                ecg_interval = None
    time.sleep(30)

Payload decoder - TTN:

function Decoder(bytes, port) {
  function bytesToFloat(bytes) {
    // JavaScript bitwise operators yield a 32 bits integer, not a float.
    // Assume LSB (least significant byte first).
    var bits = bytes[3]<<24 | bytes[2]<<16 | bytes[1]<<8 | bytes[0];
    var sign = (bits>>>31 === 0) ? 1.0 : -1.0;
    var e = bits>>>23 & 0xff;
    var m = (e === 0) ? (bits & 0x7fffff)<<1 : (bits & 0x7fffff) | 0x800000;
    var f = sign * m * Math.pow(2, e - 150);
    return f;
  }
  function bytesToShort(bytes) {
    var i = bytes[0]<<8 | bytes[1];
    return i;
  }
  var decoded = {};

  if (port === 1) {
    decoded = {
      gps: {
        lat: +bytesToFloat(bytes.slice(0, 4)),
        lng: +bytesToFloat(bytes.slice(4, 8))
      }
    }
  }
  else if (port === 2) {
    decoded = {
      ecgInterval: bytesToShort(bytes)
    }
  }
  return decoded;
}

### Transmitting the data / connectivity

I had ordered the FiPy which have Celluar connection but ended up using only LoRa. I planned to use Bluetooth between the FiPy and LoPy but revised my idea although I got Bluetooth working, because I figured if the main unit (FiPy) whent down, the LoPy would not get it's data to the internet, so I decided to use LoRaWAN on both the FiPy and LoPy indepentily.

Due to the limitations of LoRaWAN i only sends the GPS coordinates from the FiPy every 5 minutes. I used LoRaWAN with The Things Network.

Orginally I had plan to send the ECG data from the LoPy in realtime, every second, but rethinked and ended up sending the heart interval every 5 minutes. I also send the specific user logged in every 5 min.

### Presenting the data

As my orginal idea centered around a human supervised monitoring system I chose to only show the stats in real-time and don't use a database. Due to time contraints I used a simple dashboard on Ubidots.


### Finalizing the design

In the end I ran out of time and hit some limitations of specifically LoRaWAN which made my original ideas impossible. In my original idea or vision I had pictured a realtime monitoring system which updated at least every second so I could send and then visualize a ECG plot. I had to rethink a way to use the ECG meaningfully and had to compromize.

If I had a lot more resources and time I think I would have come much closer to my original vision. Replacing LoRaWAN with WLAN and/or have a central communication device or even a satelite modem (would make much more sense for my original application out in the field where no LoRaWAN would be available)

I would also have liked to do much more with the NFC part of the project, now I only use the UID to log in to my device.

I wanted originally to send the raw ECG data to the internet and a dashboard, when faced with the limitations of LoRaWAN i did look up if I could use a machine learning algorimth on the Pycom device to sense different ECG states of the agent wearing the device, but in the end it would take to much time and I decided to only send the interval between heartbeats.

I had a lot of problems with connecting to LoRa due to bad coverage so it was hard to test the system as a whole.

I also had a lot of problems with the ECG data protocol, some kind of error sometimes manifested it self when I read the UART data. This took a lot of time from the project and resulted in trying to get something basic to work.

The most successful part of the project in my mind is what lessons I have learned. If I wanted it to work and be practical I would have to find a another wireless tehcnology which have much more data allowences AND coverage. A more powerful main device to run a AI locally would benefit the project when radio coverage and/or ability to transmit would be affected.

It did also provide me with some practical knowledge about to read and process ECG data. In a more ambitious project I would have incorporated some machine learning algorithm, both on the edge devices and in a central server.




{%youtube ULU7ldsv5jw %}