[HungMH] Py-Redis Container Example

tags: IMRC Container Docker Kubernetes Redis Python Linux Documentation

Table of Content

• [HungMH] Py-Redis Container Example
  • Table of Content
  • 1. Introduction to Environment
  • 2. Basic Architecture
    • redis
    • py-redis
  • 3. Helm Charts for Redis server and Python Redis client
    • redis
    • py-redis-client
  • 4. Running the System using Helm Charts
  • 5. Running Multiple Python Redis clients
  • 6. Removing Installed Helm Charts

1. Introduction to Environment

This example is run in Hung K8s Cluster in IMRC.

By default, Redis container can only run in Linux system, so we will use one of the Ubuntu node in the cluster, namely node mink8s1-node1-u1604.

Inside you can find a py-redis directory which has the following contents.

• app contains files for building the Docker image
  • app.py simple Redis client for read/write, written in Python
  • Dockerfile for building the Docker image of the Python code
  • requirements.txt contains the package requirements for running the Python code
• helm-charts contains the Helm charts for easily deploying Python and Redis to Kubernetes
  • py-redis-client for Python-based Redis client
  • redisdb for Redis server

2. Basic Architecture

As depicted in the figure above, this example is using 2 kinds of Docker images.

redis is used for hosting the Redis server, while py-redis acts as the Python-based client for connecting to the Redis server and doing a simple read/write process.

redis

The redis image is pretty straightforward. Simply pull the Redis image from Docker hub and run.

By default, Redis can be accessed from port 6379.

py-redis

Before we can run this container, it is necessary to build the Docker container image.

First, go to the app folder and run this command.

docker build . -t py-redis:1.0

This will build the application into a Docker image with tag py-redis:1.0.

The py-redis image contains a simple Python script named app.py.
Below are the contents of app.py.

import os

from redis import Redis

#setup the variables based from environment var
REDIS_HOST = os.environ['REDIS_HOST']
REDIS_PORT = os.environ['REDIS_PORT']
REDIS_KEY = os.environ['REDIS_KEY']
REDIS_VALUE = os.environ['REDIS_VALUE']

#establish connection to redis
client = Redis(host=REDIS_HOST, port=REDIS_PORT)

# set a key-value pair in redis
client.set(REDIS_KEY, REDIS_VALUE)

# get a value from redis based on the key
value = client.get(REDIS_KEY)

# print the value
print(value)

When running the container, several environment variables must be passed, namely:

• REDIS_HOST and REDIS_PORT for connecting to the Redis server
• REDIS_KEY and REDIS_VALUE for the key-value pair to be stored in Redis server.

3. Helm Charts for Redis server and Python Redis client

Simplest way to run this example in Kubernetes is by using Helm charts.

In a Helm chart, the YAML files for deploying to Kubernetes are stored in the templates directory. The templates contains multiple placeholders whose values are determined by values.yaml.

Contents of both redisdb and py-redis-client Helm charts are not fully discussed here, but we try to point out important sections in each values.yaml.

redis

Below is a snapshot of a section in the values.yaml for Redis server.

...
node: 'mink8s1-node1-u1604'

service:
  type:
    redisdb: 'ClusterIP'
  port:
    redisdb: '6379'
  ip:
    redisdb: '10.110.20.11'
...

• node - the hostname where we want to deploy the Redis server.
• service.type.redisdb - type of Service in Kubernetes, normally unnecessary to be changed
• service.port.redisdb - port where we want to access the Redis server, will be passed to the REDIS_PORT env. variables when running the Python client
• service.ip.redisdb - cluster IP where we want to access the Redis server, will be passed to the REDIS_HOST env. variables when running the Python client

py-redis-client

Below is a snapshot of a section in the values.yaml for Python Redis client.

...
node: 'mink8s1-node1-u1604'

service:
  type:
    pyredis: 'ClusterIP'
  port:
    pyredis: '80'
  ip:
    pyredis: '10.110.20.12'
    
env:
  REDIS_HOST: '10.110.20.11'
  REDIS_PORT: '6379'
  REDIS_KEY: 'super'
  REDIS_VALUE: 'tole' 
...

• node - the hostname where we want to deploy the Python Redis client.
• service.type.redisdb - type of Service in Kubernetes, normally unnecessary to be changed
• service.port.redisdb - port where we want to access the Python Redis client
• service.ip.redisdb - cluster IP where we want to access the Python Redis client
• env.REDIS_HOST - host address of Redis server, must be matched with service.ip.redisdb
• env.REDIS_PORT - port of Redis server, must be matched with service.port.redisdb
• env.REDIS_KEY and env.REDIS_VALUE - key-value pair to be stored in Redis.

4. Running the System using Helm Charts

First, run this command to deploy the Redis server to Kubernetes using the redisdb Helm chart.

helm install redisdb redisdb

Then, run this command to deploy the Python Redis client to Kubernetes using the py-redis-client Helm chart.

helm install py-redis-client py-redis-client

Check the deployed Helm charts with this command.

helm ls

Next, we will interact directly with the Kubernetes system using kubectl.
Use this command to check the pods status.

kubectl get pods

redisdb-654c57979b-24x9d is the pod for Redis server and pyredis-578d99dffb-zspnc is the pod for Python Redis client.
Notice that the pod for Python Redis client is stuck at CrashLoopBackOff, which is normal and expected considering that the app.py script ends at showing the value from Redis.

To demonstrate that we have successfully stored a key-value pair in Redis, let's check the logs of Python Redis client pod.
Use this command to check the log of Kubernetes pod.

kubectl logs pyredis-578d99dffb-zspnc

The logs displayed tole, which is the value stored in REDIS_VALUE environment variable and set in the values.yaml as env.REDIS_VALUE.

5. Running Multiple Python Redis clients

To demonstrate the capabilities of running multiple Redis clients, we can install multiple Helm charts at the same time while passing different key-value pairs to the Redis server.

To install multiple instance of Python Redis client, you can:

1. Make multiple copies of py-redis-client Helm chart and modify the contents of values.yaml.
2. Use the same py-redis-client Helm chart but run it with different values

For this example, we will do Option 2 which is the fastest and simplest way.
Run the commands below to deploy additional 3 instances of Python Redis client.

helm install py-redis-client-1 py-redis-client --set env.REDIS_KEY=client1 --set env.REDIS_VALUE=client1 --set service.ip.pyredis=10.110.20.21

helm install py-redis-client-2 py-redis-client --set env.REDIS_KEY=client2 --set env.REDIS_VALUE=client2 --set service.ip.pyredis=10.110.20.22

helm install py-redis-client-3 py-redis-client --set env.REDIS_KEY=client3 --set env.REDIS_VALUE=client3 --set service.ip.pyredis=10.110.20.23

Notice that each Python Redis client is named differently (py-redis-client-1 to -3) with different REDIS_KEY, REDIS_VALUE, and ip.

Run kubectl get pods to get the name of the Python client pods.

Now, let's check the value of each Python client pods.

The output (client1 to client3) actually matched the values of REDIS_VALUE that we passed when running helm install.

6. Removing Installed Helm Charts

First, check the currently installed Helm charts.

helm ls

Passed the value of column NAME to delete the corresponding installation.

# to delete installed Helm charts one-by-one
helm delete py-redis-client-1 
helm delete py-redis-client-2 
helm delete py-redis-client-3
helm delete redisdb

# to delete multiple installations in one command
helm delete py-redis-client-1 py-redis-client-2 py-redis-client-3 redisdb