# Part 3: Introduction to TIG Stacks
###### tags: `TA Stuff RP2` `Raspberry Pi Pico`
The TIG (Telegraf, Influx and Grafana) Stack is an acronym for a platform of open source tools built to make collection, storage, graphing, and alerting on time series data easy. In this exercise we are going to set up a TIG-stack using Docker and Docker-compose for easy deployment.
A **time series** is simply any set of values with a timestamp where time is a meaningful component of the data. The classic real world example of a time series is stock currency exchange price data. When working with IoT-sensors it is often logged in a time series database.
Some widely used tools are:
* **Telegraf** is a metrics collection agent. Use it to collect and send metrics to InfluxDB. Telegraf’s plugin architecture supports collection of metrics from 100+ popular services right out of the box.
* **InfluxDB** is a high performance Time Series Database. It can store hundreds of thousands of points per second. The InfluxDB SQL-like query language included in **InfluxDB V1** was built specifically for time series. InfluxQL is a query language that is very similar to SQL and that allows any user to query its data and filter it. In **InfluxDB V2** and **V3**, they make use of their new language **Flux**, which you can read more about [here](https://docs.influxdata.com/flux/v0/). Read more about InfluxDB in their documentation: [InfluxDB Documentation](https://docs.influxdata.com/)
* **Grafana** is an open-source platform for data visualization, monitoring and analysis. In Grafana, users can to create dashboards with panels, each representing specific metrics over a set time-frame. Grafana supports graph, table, heatmap and freetext panels.
In this exercise we will use the containers platform [Docker](https://www.docker.com/). Docker can be easily installed in in a large variety of platforms like a [Raspberry Pi.](https://www.raspberrypi.org/blog/docker-comes-to-raspberry-pi/)
[Install Docker](https://docs.docker.com/compose/install/) for your operating system. Please note, if you are using Linux you need to manually install docker-compose, on desktop systems like Docker Desktop for Mac and Windows, Docker Compose is included as part of those desktop installs.
We will use the following images:
* https://hub.docker.com/_/telegraf
* https://hub.docker.com/_/influxdb
* https://hub.docker.com/r/grafana/grafana
The images will be pulled automatically when running the docker-compose script, but they can also be manually pulled (and updated) with the command below:
```bash
$ docker pull telegraf
$ docker pull influxdb
$ docker pull grafana/grafana
```
## Setting up Mosquitto
There are several ways to set up an MQTT broker. In this guide, we will focus on a local install of Mosquitto on a Raspberry Pi running Raspberry Pi OS.
For **Windows** or **Mac**, see the Eclipse Mosquitto Download page [here](https://mosquitto.org/download/). There are installation guides for respective operating system in the README.
An alternative to Mosquitto is **HiveMQ**. Read more about HiveMQ and how to install it [here](https://www.hivemq.com/).
### Update System Packages and Installing Mosquitto
**Step 1:** Start by updating your system packages:
```=bash
$ sudo apt-get update -y
$ sudo apt-get upgrade -y
```
**Step 2:** Install Mosquitto
```=bash
$ sudo apt-get install mosquitto mosquitto-clients -y
```
**Step 3:** Set up user authentication
To make your broker secure, you need to set it up with a username and password. First, create a password file:
```=bash
$ sudo mosquitto_passwd -c /etc/mosquitto/passwd username # Replace username with a name of your choice
```
**Step 4:** Edit the config file to allow your credentials:
```=bash
$ sudo nano /etc/mosquitto/mosquitto.conf
```
Add the following lines to the config file:
```=bash
allow_anonymous false
password_file /etc/mosquitto/passwd
```
Then restart the Mosquitto service:
```=bash
$ sudo systemctl restart mosquitto
```
**Step 5:** Test your Mosquitto broker
```=bash
$ mosquitto_sub -h localhost -t test -u "username" -P "password" # Replace with your username and password
$ mosquitto_pub -h localhost -t test -m "hello world" -u "username" -P "password" # Replace with your username and password
```
### Docker-compose
Compose is a tool for defining and running multi-container Docker applications, which in this case is three different containers. Instead of firing up all container individually with the respective config, we can write a script which does this for us. A YAML file is used to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.
Start all services with `docker-compose up`, and take them down with `docker-compose down`. Very easy.
**Step 1:** Create a file `docker-compose.yml` in a working folder.
**Step 2:** Use this template and add it to your `docker-compose.yml` file.
Make sure to read through the template **thoroughly** and get an understanding of what needs to be altered later on before proceeding.
```=bash
#version: '3.9'
networks:
tig-net:
driver: bridge
volumes:
tig-data:
services:
influxdb:
image: influxdb:latest
container_name: influxdb
ports:
- 8086:8086
environment:
INFLUXDB_HTTP_AUTH_ENABLED: "true"
INFLUXDB_DB: "" # THIS WILL BE REPLACED
INFLUXDB_ADMIN_USER: "" # THIS WILL BE REPLACED
INFLUXDB_ADMIN_PASSWORD: "" # THIS WILL BE REPLACED
networks:
- tig-net
volumes:
- tig-data:/var/lib/influxdb
grafana:
image: grafana/grafana:latest
container_name: grafana
ports:
- 3000:3000
environment:
GF_SECURITY_ADMIN_USER: {GRAFANA_USER_NAME} # Replace this before starting the containers
GF_SECURITY_ADMIN_PASSWORD: {GRAFANA_PASSWORD} # Replace this before starting the containers
networks:
- tig-net
volumes:
- tig-data:/var/lib/grafana
restart: always
telegraf:
image: telegraf:latest
depends_on:
- "influxdb"
environment:
HOST_NAME: "telegraf"
INFLUXDB_HOST: "influxdb"
INFLUXDB_PORT: "8086"
DATABASE: "" # THIS WILL BE REPLACED
volumes:
- ./telegraf.conf:/etc/telegraf/telegraf.conf
tty: true
networks:
- tig-net
privileged: true
```
Take note of this part, which you will have to alter **before** you start the containers. You can choose whatever username and password you want. These credentials are used to log in to Grafana. Which you can do on **127.0.1:3000**.
```=bash
environment:
GF_SECURITY_ADMIN_USER: {GRAFANA_USER_NAME} # Replace this before starting the containers
GF_SECURITY_ADMIN_PASSWORD: {GRAFANA_PASSWORD} # Replace this before starting the containers
```
**Step 3:** Go to **127.0.1:8086** in your web browser, and follow the setup guide for InfluxDB:
Be sure to save your **API token** for later.
Once having done this, stop the containers with `sudo docker compose down` and add your credentials to your `docker-compose.yml` file before starting the containers with `sudo docker compose up` again
Replace the `INFLUX_DB` with the name of the **Bucket** you created in the InfluxDB setup process, and replace `{USERNAME}` and `{PASSWORD}` with the user name and password you chose in the setup process.
```=bash
environment:
INFLUXDB_HTTP_AUTH_ENABLED: "true"
INFLUXDB_DB: "{BUCKET_NAME}" # THIS WILL BE REPLACED
INFLUXDB_ADMIN_USER: "{USERNAME}" # THIS WILL BE REPLACED
INFLUXDB_ADMIN_PASSWORD: "{PASSWORD}" # THIS WILL BE REPLACED
```
## Configuring Telegraf
You will need to configure Telegraf.
Study the template below and put it in a filed called `telegraf.conf` in the same working directory as your `docker-compose.yml` file.
Replace the corresponding variables to fit your personal setup.
```=bash
[agent]
flush_interval = "30s"
interval = "30s"
[[inputs.mqtt_consumer]]
name_override = "{}" # Choose a name
servers = ["tcp://{IP}:1883"] # REPLACE WITH IP OF YOUR BROKER
qos = 0
connection_timeout = "30s"
topics = [ "{TOPIC1}", "{TOPIC2}", "TOPIC.." ] # REPLACE WITH YOUR OWN TOPICS
username = "{USERNAME}" # REPLACE THIS WITH MOSQUITTO USERNAME
password = "{PASSWORD}" # REPLACE THIS WITH MOSQUITTO PASSWORD
data_format = "value" # MIGHT NEED REPLACING
data_type = "float" # MIGHT NEED REPLACING
[[outputs.influxdb_v2]]
bucket = "{BUCKETNAME}" # REPLACE WITH BUCKET NAME
urls = [ "http://influxdb:8086" ]
token = "{API-TOKEN}" # REPLACE WITH InfluxDB API TOKEN
organization = "{ORGANIZATION}" # REPLACE WITH ORGANIZATION NAME
```
## Setting up Grafana
Go to **127.0.1:3000** in your browser, which will take you to Grafana.
Log in with the credentials you set in the `docker-compose.yml` file.
**Step 1:** Install the InfluxDB plugin:
**Step 2:** Add a data source:
**Step 3:** Configure your data source:
The values to be entered should be self explanatory, and will be the same as the ones you used in the previous steps.
**Note:** You can choose between **InfluxQL** and **Flux** according to your liking.
**Step 4:** Create a new dashboard:
**Step 5:** Add a new panel:
Take note of the **query**, in this example **Flux** is used. These queries must be altered to fit your individual setup.
**Step 6:** You should now be ready for creating your own personal dashboard!
<style>
html, body, .ui-content {
background-color: #333;
color: #ddd;
}
body > .ui-infobar {
display: none;
}
.ui-view-area > .ui-infobar {
display: block;
}
.markdown-body h1,
.markdown-body h2,
.markdown-body h3,
.markdown-body h4,
.markdown-body h5,
.markdown-body h6 {
color: #ddd;
}
.markdown-body h1,
.markdown-body h2 {
border-bottom-color: #ffffff69;
}
.markdown-body h1 .octicon-link,
.markdown-body h2 .octicon-link,
.markdown-body h3 .octicon-link,
.markdown-body h4 .octicon-link,
.markdown-body h5 .octicon-link,
.markdown-body h6 .octicon-link {
color: #fff;
}
.markdown-body img {
background-color: transparent;
}
.ui-toc-dropdown .nav>.active:focus>a, .ui-toc-dropdown .nav>.active:hover>a, .ui-toc-dropdown .nav>.active>a {
color: white;
border-left: 2px solid white;
}
.expand-toggle:hover,
.expand-toggle:focus,
.back-to-top:hover,
.back-to-top:focus,
.go-to-bottom:hover,
.go-to-bottom:focus {
color: white;
}
.ui-toc-dropdown {
background-color: #333;
}
.ui-toc-label.btn {
background-color: #191919;
color: white;
}
.ui-toc-dropdown .nav>li>a:focus,
.ui-toc-dropdown .nav>li>a:hover {
color: white;
border-left: 1px solid white;
}
.markdown-body blockquote {
color: #fff;
}
.markdown-body table tr {
background-color: #5f5f5f;
}
.markdown-body table tr:nth-child(2n) {
background-color: #4f4f4f;
}
.markdown-body code,
.markdown-body tt {
color: #f;
background-color: rgba(230, 230, 230, 0.36);
}
a,
.open-files-container li.selected a {
color: #5EB7E0;
}
.markdown-body pre {
color: black;
}
</style>
Sign in with Wallet
Connect another wallet