**Topic: OAI and M5G over Kubernetes**
> University: Bandung Institute of Technology
> Department: School of Electrical Engineering and Informatics (Telecommunication Engineering)
> Email: [e-mail] [color=#ffa700]

# **<center><i class=""></i> Learn about Redis</center>**
Contents:
[TOC]
## What is Redis?
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams.
Redis also supports trivial-to-setup master-slave asynchronous replication, with very fast non-blocking first synchronization, auto-reconnection with partial resynchronization on net split.

Other features include:
* Transactions
* Pub/Sub
* Lua scripting
* Keys with a limited time-to-live
* LRU eviction of keys
* Automatic failover

## Installing Redis
Download the Redis installation package from https://redis.io/download. Then just extract and set the path to the Redis directory in the environment variable. You can be seen in the following steps:

$ wget http://download.redis.io/releases/redis-2.8.19.tar.gz
$ tar xzf redis-2.8.19.tar.gz
$ cd redis-2.8.19
$ make
$ make test


$ sudo make install


## Install Redis Server
https://www.interserver.net/tips/kb/install-configure-redis-ubuntu/

It’s good that we can now install Redis directly. It means, we don’t have to download the source and then compile it in order to install Redis. To install Redis server, execute the following command in terminal.

$sudo apt-get install redis-server -y


It will take few seconds to install Redis server on Ubuntu 18.04 VM. The -y flag we used is to avoid getting a confirmation message to install Redis server.

Once the Redis server is installed on your machine, execute the following command to check if Redis is working properly or not.

$redis-cli
If Redis server is installed successfully, the redis-cli command will open a command line interface for Redis. To test if Redis is working correctly or not, Execute the following command.

$127.0.0.1:6379> PING


## Configure Redis Server

Redis configuration file is located at “/etc/redis/redis.conf”. To edit the configuration file using nano, execute the following command in terminal.

$sudo nano /etc/redis/redis.conf
Now, you will see a redis configuration file with lots of directives and information. Press CTRL+W to enable search interface and type “# maxmemory” and then hit the Enter key to find the line containing “# maxmemory”.

Once you find the line containing following content:

maxmemory <bytes>
Replace it with:

maxmemory 256mb
Once you find the line containing following content:

requirepass foobared
Replace it with:

requirepass YOUR_DESIRED_REDIS_PASSWORD

Restart the Redis server

$sudo service redis restart

And then check the status of the Redis service by executing the command given below. It should show “Running”.

$sudo service redis status


## Configuring Redis

Open:

sudo vim /etc/redis/redis.conf

Editing service `/lib/systemd/system/redis-server.service` and replace with:

[Unit]
Description=Advanced key-value store
After=network.target
Documentation=http://redis.io/documentation, man:redis-server(1)

[Service]
Type=forking
ExecStart=/usr/bin/redis-server /etc/redis/redis.conf
PIDFile=/var/run/redis/redis-server.pid
TimeoutStopSec=0
Restart=always
User=redis
Group=redis

ExecStartPre=-/bin/run-parts --verbose /etc/redis/redis-server.pre-up.d
ExecStartPost=-/bin/run-parts --verbose /etc/redis/redis-server.post-up.d
ExecStop=-/bin/run-parts --verbose /etc/redis/redis-server.pre-down.d
ExecStop=/bin/kill -s TERM $MAINPID
ExecStopPost=-/bin/run-parts --verbose /etc/redis/redis-server.post-down.d

UMask=007
PrivateTmp=yes
PrivateDevices=yes
ProtectHome=yes
ReadOnlyDirectories=/
ReadWriteDirectories=-/var/lib/redis
ReadWriteDirectories=-/var/log/redis
ReadWriteDirectories=-/var/run/redis
CapabilityBoundingSet=~CAP_SYS_PTRACE

# redis-server writes its own config file when in cluster mode so we allow
# writing there (NB. ProtectSystem=true over ProtectSystem=full)
ProtectSystem=true
ReadWriteDirectories=-/etc/redis

[Install]
WantedBy=multi-user.target
Alias=redis.service

Restart Redis:

systemctl restart redis-server.service

Status:

systemctl status redis-server.service

Run redis-server:

redis-server

Try to ping in redis-cli:


-------------------------------

# **<center><i class=""></i> Learn about Elastic Search</center>**

## What is Elastic Search?
Elasticsearch is a highly scalable open-source full-text search and analytics engine. It allows you to store, search, and analyze big volumes of data quickly and in near real time. It is generally used as the underlying engine/technology that powers applications that have complex search features and requirements. Elasticsearch provides a distributed system on top of Lucene StandardAnalyzer for indexing and automatic type guessing and utilizes a JSON based REST API to refer to Lucene features.
It is easy to set up out of the box since it ships with sensible defaults and hides complexity from beginners. It has a short learning curve to grasp the basics so anyone with a bit of efforts can become productive very quickly. It is schema-less, using some defaults to index the data.
## Backend Components
To better understand Elasticsearch and its usage is good to have a general understanding of the main backend components.
**Node**
A node is a single server that is part of a cluster, stores our data, and participates in the cluster’s indexing and search capabilities. Just like a cluster, a node is identified by a name which by default is a random Universally Unique Identifier (UUID) that is assigned to the node at startup. We can edit the default node names in case we want to.
**Cluster**
A cluster is a collection of one or more nodes that together holds your entire data and provides federated indexing and search capabilities. There can be N nodes with the same cluster name.
Elasticsearch operates in a distributed environment: with cross-cluster replication, a secondary cluster can spring into action as a hot backup.
**Index**
The index is a collection of documents that have similar characteristics. For example, we can have an index for a specific customer, another for a product information, and another for a different typology of data. An index is identified by a unique name that refers to the index when performing indexing search, update, and delete operations. In a single cluster, we can define as many indexes as we want. Index is similiar to database in an RDBMS.
**Document**
A document is a basic unit of information that can be indexed. For example, you can have an index about your product and then a document for a single customer. This document is expressed in JSON (JavaScript Object Notation) which is a ubiquitous internet data interchange format. Analogy to a single raw in a DB.
Within an index, you can store as many documents as you want, so that in the same index you can have a document for a single product, and yet another for a single order.
**Shard and Replicas**
Elasticsearch provides the ability to subdivide your index into multiple pieces called shards. When you create an index, you can simply define the number of shards that you want. Each shard is in itself a fully-functional and independent “index” that can be hosted on any node in the cluster.
Shards is important cause it allows to horizontally split your data volume, potentially also in multiple nodes paralelizing operations thus increasing performance. Shards can also be used by making multiple copies of your index into replicas shards, which in cloud environments could be useful to provide high availability.

## Installing Elasticsearch

Download for linux 18.04: https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.2-linux-x86_64.tar.gz

Move and extract elasticsearch to '$HOME':

tar -xzvf elasticsearch-7.5.2-linux-x86_64.tar.gz
cd elasticsearch-7.5.2
./bin/elasticsearch


If we want to check, open the browser:

localhost:9200


### Configuring Elasticsearch

I will use Visual Studio Code, because VScode has more userfriendly environtment when we struggling for any code than UNIX terminal/vim/nano/pico.

In realistic production we will need more than 1 node to implement inside the cluster using odd number of nodes. So, now I'll show how to create 3 nodes of elasticsearch.

Step 1: I will create directory `elasticsearch-cluster`

sudo mkdir elasticsearch-cluster

Step 2: Copying stable version of elasticsearch into the `elasticsearch-cluster` Folder:

sudo mv elasticsearch-7.5.2 elasticsearch-cluster/

Step 3: Clone Stable version and rename it with additional name (e.g., node-1, node-2, node-3) so it will be our nodes:

sudo cp -avr elasticsearch-7.5.2-node1/ elasticsearch-7.5.2-node2/
sudo cp -avr elasticsearch-7.5.2-node1/ elasticsearch-7.5.2-node3/

Result:

#### Configuration each node

Open Those node folder by Visual Studio Code.

**NODE 1**

For each elasticsearch node, open the config folder, and open `elasticsearch.yml`:

# ======================== Elasticsearch Configuration =========================

# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-cluster
#
# ------------------------------------ Node ------------------------------------

node.name: node-1

# ----------------------------------- Paths ------------------------------------

# ----------------------------------- Memory -----------------------------------

bootstrap.memory_lock: true


# ---------------------------------- Network -----------------------------------

http.port: 9200

# --------------------------------- Discovery ----------------------------------


cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]

# ---------------------------------- Gateway -----------------------------------

# ---------------------------------- Various -----------------------------------

**NODE 2**

# ======================== Elasticsearch Configuration =========================

# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-cluster
#
# ------------------------------------ Node ------------------------------------

node.name: node-2

# ----------------------------------- Paths ------------------------------------

# ----------------------------------- Memory -----------------------------------

bootstrap.memory_lock: true


# ---------------------------------- Network -----------------------------------

http.port: 9201

# --------------------------------- Discovery ----------------------------------


cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]

# ---------------------------------- Gateway -----------------------------------

# ---------------------------------- Various -----------------------------------

**NODE 3**

# ======================== Elasticsearch Configuration =========================

# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-cluster
#
# ------------------------------------ Node ------------------------------------

node.name: node-3

# ----------------------------------- Paths ------------------------------------

# ----------------------------------- Memory -----------------------------------

bootstrap.memory_lock: true


# ---------------------------------- Network -----------------------------------

http.port: 9202

# --------------------------------- Discovery ----------------------------------


cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]

# ---------------------------------- Gateway -----------------------------------

# ---------------------------------- Various -----------------------------------

Open 3 terminal tab to run those nodes:

cd $HOME/elasticsearch-cluster/elasticsearch-7.5.2-node1
./bin/elasticsearch

Run those command also for node-2 and node-3


Open browser For node-1 node-2 and node-3:

localhost:9200

localhost:9201


localhost:9202


If we browse the nodes, for example I will use node-3 to browse:

localhost:9201/_cat/nodes?v

As we can see role master (*) hold by node-1 It meant node-1 is currently the node master.