# DIOTP ## C5 - Time Series --- # Motivation Store event-based data efficiently --- ## Time series data... - Group of datas collected in different points of time. - Sequence of observations on a certain period. - **Univariate time series:** One variable varying over time. e.g. temperature measurements of a given location. - **Multivariate time series:** Multible variables varying over time. e.g. temperature, humidity and pressure vary simultaneously. --- ## ...Time series data Regular data interval is often expected. In reality some of the data can't be measured regularly, so there must be support for irregular time series too. - **Regular time series:** is collected on a constant interval and is often called **metrics**. - **Irregular time series:** has a varying interval(occurs irregularly) and is called **events**. [series-regular-irregular-time](https://www.ibm.com/docs/en/streams/5.3?topic=series-regular-irregular-time) --- ## Metric-based data - **Characteristics:** Quantative, Time-series, Continuous - **Examples:** - **System metrics:** CPU Utilization, MEM Usage, disk I/O, network traffic - **Financial metrics:** Stock price, exchange rates, trading volumes --- ## Event-based data - **Characteristics:** Qualitative or Quantative, Irregular - **Examples:** - **System events:** Program crashes, network outages... - **Financial events:** buy/sell orders, deposits/withdrawals... --- # InfluxDB Time-series Database --- ## InfluxDB - Search-engine database for searching results - Document databases are optimized for handling documents - Time Series database for timeline search [influxdata.com](https://www.influxdata.com/index/) --- ## InfluxDB Line Protocol (v1.x) InfluxDB uses line protocol to write data points. It is a text-based format that provides the measurement, tag set, field set, and timestamp of a data point. Syntax: ``` measurementName,tagKey=tagValue fieldKey="fieldValue" 1465839830100400200 --------------- --------------- --------------------- ------------------- | | | | Measurement Tag set Field set Timestamp ``` --- ## ...InfluxDB Line Protocol (v1.x) Example: ``` myMeasurement,tag1=value1,tag2=value2 fieldKey="fieldValue" 1556813561098000000 ``` Lines separated by the newline character "`\n`" represent a single point in InfluxDB. Line protocol is whitespace sensitive. ... --- ## InfluxDB HTTP API ... to be detailed --- # Setting up InfluxDB Prerequisites: - Remote Ubuntu22.04 LTS - Remote access - Admin privileges --- ## Setup SSH Config SSH Directory `~/.ssh` configure `config` file ```bash! nano ~/.ssh/config ``` In the config, define SSH configurations ```txt Host diotp-vm HostName public.ip.addr.here User remote_user IdentityFile ~/.ssh/keyfile ``` Login: ```bash ssh diotp-vm ``` --- ## Install InfluxDB [Install and setup InfluxDB on Ubuntu 20.04|22.04](https://medium.com/yavar/install-and-setup-influxdb-on-ubuntu-20-04-22-04-3d6e090ec70c) ```bash! # Set influxdata repository curl --silent --location -O \ https://repos.influxdata.com/influxdata-archive.key echo "943666881a1b8d9b849b74caebf02d3465d6beb716510d86a39f6c8e8dac7515 influxdata-archive.key" \ | sha256sum --check - && cat influxdata-archive.key | sudo gpg --dearmor \ | sudo tee /etc/apt/trusted.gpg.d/influxdata-archive.gpg > /dev/null \ && echo 'deb [signed-by=/etc/apt/trusted.gpg.d/influxdata-archive.gpg] https://repos.influxdata.com/debian stable main' | sudo tee /etc/apt/sources.list.d/influxdata.list # Install influxdb sudo apt update && sudo apt install -y influxdb2 influx version sudo systemctl enable influxdb sudo systemctl start influxdb sudo systemctl status influxdb ``` --- ## Configure InfluxDB Edit configuration ```bash # Modify InfluxDB configuration sudo nano /etc/influx/influxdb.conf # Restart InfluxDB service sudo systemctl restart influxdb # Check InfluxDB status sudo systemctl status influxdb ``` --- ## Tunnel to dashboard Command to setup tunnel for InfluxDB Dashboard(port must be corrected) ```bash ssh -f -N -L 8086:localhost:8086 remote_user@remote_server ``` Same as above, but in SSH_CONFIG: ```bash Host my_remote_server # The hostname or IP address of the remote server HostName remote_server # Your SSH username for the remote server User user # Set up local port forwarding LocalForward 8086 localhost:8086 ``` --- ## Start SSH Tunnel for Dashboard ```bash ssh diotp-vm-influx ``` The Influx Dashboard should be available in `localhost:8086`. Use browser. --- ## Setup InfluxDB [Documentation](https://docs.influxdata.com/influxdb/v2/get-started/setup/) --- ### S1 - Run initial setup process [Documentation](https://docs.influxdata.com/influxdb/v2/get-started/setup/?t=Set+up+with+the+CLI#run-initial-setup-process) Options: - **Via UI** (first time) - Via CLI - Via HTTP API --- ### S2 - Create an all access API token [Documentation](https://docs.influxdata.com/influxdb/v2/get-started/setup/?t=Set+up+with+the+CLI#create-an-all-access-api-token) --- ### S3 - Configure authentication credentials [Documentation](https://docs.influxdata.com/influxdb/v2/get-started/setup/?t=Set+up+with+the+CLI#configure-authentication-credentials) --- ### S4 - Create a bucket [Documentation](https://docs.influxdata.com/influxdb/v2/get-started/setup/?t=Set+up+with+the+CLI#create-a-bucket) --- # Write time-series data [Write methods](https://docs.influxdata.com/influxdb/cloud-serverless/get-started/write/): - Influx user interface (UI) - Influx CLI - InfluxDB HTTP API (v1 and v2) --- ### Influx user interface (UI) [Influx UI](https://docs.influxdata.com/influxdb/cloud-serverless/get-started/write/#line-protocol-elements) --- ### Influx CLI [Influx CLI](https://docs.influxdata.com/influxdb/cloud-serverless/get-started/write/?t=influx+CLI#line-protocol-elements) ```bash! influx ... ``` --- ### InfluxDB HTTP API v2 [HTTP API v2](https://docs.influxdata.com/influxdb/cloud-serverless/get-started/write/?t=v2+API#line-protocol-elements) --- # Query time-series data [Documentation](https://docs.influxdata.com/influxdb/cloud-serverless/get-started/query/) --- ## Query data with Flux [Documentation](https://docs.influxdata.com/influxdb/v2/get-started/query/#query-data-with-flux) Example query: ```js from(bucket: "get-started") |> range(start: 2022-01-01T08:00:00Z, stop: 2022-01-01T20:00:01Z) |> filter(fn: (r) => r._measurement == "home") |> filter(fn: (r) => r._field== "co" or r._field == "hum" or r._field == "temp") ``` --- ## Visualize data in Dashboard [Documentation](https://docs.influxdata.com/influxdb/v2/visualize-data/)  --- ## Visualization types [Documentation](https://docs.influxdata.com/influxdb/v2/visualize-data/visualization-types/). Important ones: - Gauge - Graph --- ### Gauge  The Gauge visualization displays the most recent value for a time series in a gauge. --- ### Graph  The Graph visualization provides several types of graphs, each configured through the Graph controls. ---