# Part 3: Datacake Integration
###### tags: `TA Stuff RP2` `Raspberry Pi Pico`

## LoRaWAN

In this tutorial, we'll be focusing on sending data to Datacake for storage and visualization.
This tutorial will be mostly focusing on integrating Datacake with Helium and HTTP, but the steps should be very similar for other providers.

---
:::danger
**⚠️ Important ⚠️ Make sure the LoRa antenna is connected properly before running any LoRa code on your device. Not doing so might break your device.** You can find how to properly connect the antenna [here](#Connect-the-antenna).
:::

---

## Table of Contents
[TOC]

## Getting started

### Prerequisites
1. Make sure you have an account on Helium, TTN, or any similar provider if using LoRa.
2. Make sure you are able to connect to the network and send data.
3. If using HTTP, make sure you are able to connect to a WiFi network.

Check the previous tutorials for more details and help on how to use Helium, TTN and WiFi.

## Registration

### 1. Signing up
---
Before you can start using Datacake, you need to [create an account](https://app.datacake.de/signup). Once you have done that, you should be greeted with the dashboard.


### 2. Adding a device
---
The next step is to add a device. Click on the blue "Add device" button on the top right and you should get a popup screen.

If you are using LoRaWan, pick LoRaWan


Otherwise if you're using HTTP or MQTT, pick API


At this step pick "New Product" and give the product a name. You can also search through the product templates to see if you can find an already existing template.

If you picked LoRaWan, it should look like this:


If you picked API, it should look like this:


At this step, choose the network server you will be using for LoRa! As mentioned we will be showing Helium in this tutorial but the process should be similar for other providers.


At this step, you need to provide the LoRa DEV_EUI you got either from the Helium/TTN/etc.
:::warning
Note that filling the DEV_EUI as "FF FF FF FF FF FF FF FF" will not work.
:::
Give the device a name and click next.


For HTTP, we choose a unique identifier for the device, along with a tag and a location for it. You are free to either generate one or make a custom one yourself.


At this step, you can choose the plan that suits you best. We suggest that you go with the free plan until you have configured your devices and feel comfortable going with a paid plan and have a need for it. This is the same for both HTTP and LoRaWAN.


Once you follow all the steps above you should get a success message and see that a device with the name you gave is added to the dashboard

Helium:


HTTP (Pico):


## Integration:

Now that you have a device configured you can [click here](https://docs.helium.com/use-the-network/console/integrations/datacake/) and follow the Helium tutorial on how to add Datacake integration.

Then you can head to flows and add a flow similar to what you can see in the picture below.


You can also read more about flows [here.](https://docs.helium.com/use-the-network/console/flows/)

The process for TTN should be similar. See [here](https://www.thethingsindustries.com/docs/integrations/cloud-integrations/datacake/) for example.

## Configuration

There are som configurations that need to be adjusted. Head to the configuration tab by clicking on the device name and then choosing the configuration tab.

Helium:


HTTP:


Scrolling down all the way would show two sections: The payload decoder, used to format the data received, and the Fields sections where you can specify the data that will be sent to Datacake. Throughout this tutorial, we will pretend that we are receiving data from a DHT11/DHT22 sensor where there will be a floating-point temperature value and a floating-point humidity value.


### 1. Adding data fields
---
The data fields we need to add are one for the temperature and another for the humidity. Start by clicking on the "Add Field" button in the Fields section.

Fill in the information for the temperature as shown in the picture below.


* We set the type to "Float" which stands for a floating-point value, a data type used to store decimal values.
* We set the name to indicate the value stored, which we call temperature in this case.
* The identifier will be automatically generated but can be changed. If you think it should be changed, change it now because you won't be able to modifiy it without re-creating the field.
* The unit is set to degrees Celsius.
* The role can specify a role for the field, but we leave it for none for now.
* The formula could be useful to generate data fields based on other field values.

We do the same thing for humidity but with the appropriate values:


Now the fields configuration is done!

### 2. Payload Decoder
---

The idea behind the payload decoder is to allow you as the user to send the data in a custom format (encoding) and then specify for Datacake what needs to be done to retrieve the actual values (decoding).

This is very useful when using a limited bandwidth since you want to send as little data as possible to reduce the amount of data used as well as the amount of the data transmitted since large data is harder to transmit without corruption and take longer to transmit.

The Datacake payload decoder consists of a javascript function that takes a payload which is a byte array and a port number. The user can then write custom javascript code to extract the values from the payload and then return the data values as a pair of field names and value.

You can read more about it [here.](https://docs.datacake.de/lorawan/payload-decoders)

Let us take an example in the case of temperature and humidity:

#### LoRaWAN

A float can be stored in 4 or 8 bytes of data. We need two floats for the temperature and humidity. That is at least 8 bytes.

However, since we only require a decimal precision of 2 digits or maybe even less we can optimize the values we send to be stored in a single byte.

Furthermore, we know that humidity will be in the range between 0 and 100 percent while the temperature can go to the negative or positive parts but should not go higher than two digits + two decimal places. All and all if we ignore decimal separators we should be able to store the temperature and humidity as a single number of max 8 digits.

For example, we can store a temperature of 25.16 and humidity of 36.52 as the two integers: **2516, 3652**

The integers can be stored in only 4 bytes and can then be decoded using the payload decoder to get back the float temperature value.
This way we have effectively halved the amount of data sent on each transmission.

:::info
~~This method does not handle negative values or a humidity value of 100. This is left for you to solve.~~
:::

To create the integers (in Python) we could write code similar to this:

# Example temperature (in Celsius) and humidity (%) values with a negative temperature
temperature, humidity = -14.2, 42.5

# Convert the float values to integers by multiplying them by a factor (example: 10)
temp_int = int(temperature * 10)
humidity_int = int(humidity * 10)

payload = struct.pack(">hH", temp_int, humidity_int)
payload = binascii.hexlify(payload).decode("utf-8")

Now we can utilize the payload decoder to revert the process and extract the two float values for temperature and humidity by writing something similar to the following javascript code:

function Decoder(payload, port) {
    return [{
            field: "TEMPERATURE",
            // To view negative values, we shift bits.
            value: (payload[0] << 24 >> 16 | payload[1]) / 10
        },
        {
            field: "HUMIDITY",
            value: ((payload[2] << 8) | payload[3]) / 10
        }
    ];
}

#### HTTPS

With HTTPS, we don't worry so much about the space requirements. Data is sent in the JSON format, which means using dictionaries in Python. We will be using the [requests](https://requests.readthedocs.io/en/latest/) builtin Python library for this to make things easy on ourselves. To create the data, you would read the sensor data, such as the temperature and humidity values, then populate a dictionary with the values. You also need the device from the API endpoint. To send the values, you would have code similar to this:

from requests import post, Response

            ...

def http_post(url: str, data: dict[str, str|int]) -> None:
    response: Response = None
    try:
        response = post(url, json = data)
        print(response.content)
        
    except Exception as e:
        print(e)

connect()

data: dict[str, str|float] = {
    "device": "7f4a0837-ec6e-41e9-816c-e32733b3cb01",
    "temperature": 25.0,
    "humidity": 0.5
}

url: str = "https://api.datacake.co/integrations/api/5530..."

http_post(url = "https://api.datacake.co/integrations/api/5530cc92...",data = data)

It's important that in the line containing

response = post(url, json = data)

You specify the data as 'json', otherwise it may be misinterpretted by the API endpoint.

The decoder in datacake would look something like this, for simple temperature and humidity values:

function Decoder(request) {

    var payload = JSON.parse(request.body)
    var serialNumber = payload.device
    var temp
    var hum

    try {
        
        var temp = payload.temperature
        var hum = payload.humidity
        
    } catch (e) {
        
        console.log(JSON.stringify(e))
        console.log("Error parsing Configuration Field")

    }
    
    return [
        {
          device: serialNumber,
          field: "TEMPERATURE",
          value: temp
        },
        {
          device: serialNumber,
          field: "HUMIDITY",
          value: hum
        }
    ]
}

:::info
Notice how the field name in the return block is using the field identifier that we got when creating the data fields.
:::

Now when sending an encoded integer to Helium we should receive it in Datacake decoded into separate temperature and humidity values. We can check that by going to the debug tab.



And as expected the size of the payload sent from Helium to Datacake is 4 bytes.


The same should apply for data sent through the TTN network

## Building a Dashboard

The last thing to do is to start creating a dashboard to visualize the data that we get from the device. To do so head to the dashboard tab for the device you added.

You then have to toggle the edit switch on the top right to get access to the dashboard editing tools.


Click on the add widget button that showed up next to the switch you just toggled.


From the list of widgets you can pick and choose which widgets fit the data you have as well as the way you want to visualize it.


For now, let's pick the value widget and go to the Data tab.
From there we can pick the temperature field we added earlier as well as specify the unit as "Celsius". We also want to keep the decimal places as 2.


Click save and then repeat the same thing for humidity.


Now you should have two widgets that you can resize and move around to make different arrangements. Once done you can toggle the editing switch back to off to lock the widgets in place.

Once you do that last step you have completed this tutorial.

<style>

html, body, .ui-content {
background-color: #333;
color: #ddd;
}

body > .ui-infobar {
display: none;
}

.ui-view-area > .ui-infobar {
display: block;
}

.markdown-body h1,
.markdown-body h2,
.markdown-body h3,
.markdown-body h4,
.markdown-body h5,
.markdown-body h6 {
color: #ddd;
}

.markdown-body h1,
.markdown-body h2 {
border-bottom-color: #ffffff69;
}

.markdown-body h1 .octicon-link,
.markdown-body h2 .octicon-link,
.markdown-body h3 .octicon-link,
.markdown-body h4 .octicon-link,
.markdown-body h5 .octicon-link,
.markdown-body h6 .octicon-link {
color: #fff;
}

.markdown-body img {
background-color: transparent;
}

.ui-toc-dropdown .nav>.active:focus>a, .ui-toc-dropdown .nav>.active:hover>a, .ui-toc-dropdown .nav>.active>a {
color: white;
border-left: 2px solid white;
}

.expand-toggle:hover,
.expand-toggle:focus,
.back-to-top:hover,
.back-to-top:focus,
.go-to-bottom:hover,
.go-to-bottom:focus {
color: white;
}

.ui-toc-dropdown {
background-color: #333;
}

.ui-toc-label.btn {
background-color: #191919;
color: white;
}

.ui-toc-dropdown .nav>li>a:focus,
.ui-toc-dropdown .nav>li>a:hover {
color: white;
border-left: 1px solid white;
}

.markdown-body blockquote {
color: #fff;
}

.markdown-body table tr {
background-color: #5f5f5f;
}

.markdown-body table tr:nth-child(2n) {
background-color: #4f4f4f;
}

.markdown-body code,
.markdown-body tt {
color: #f;
background-color: rgba(230, 230, 230, 0.36);
}

a,
.open-files-container li.selected a {
color: #5EB7E0;
}

.markdown-body pre {
color: black;
}
</style>