Package testing for Gluster releases using docker

This details using docker to setup gluster containers with CentOS test builds, to validate packages are fine and basic functionality passes.

Setup docker

On a fresh Fedora (26 is what was used), run the following to setup docker:

#dnf install docker
#mkdir /d/docker
#rmdir /var/lib/docker/
#ln -s /d/docker /var/lib/docker
#vi /etc/sysconfig/docker
    - Remove selinux enforcement for things to work right!
#systemctl enable docker
#systemctl start docker
#docker run hello-world

The above does a few specific things that may not be needed for other installations, these (and reasons for the same) are detailed below,

• /var/lib/docker is soft linked to a different directory (in the case above /d/docker)
  • This is done, so that all docker images and work do not use up space in the root partition (which is where /var/lib was in this case), and hence before even starting docker, this was soft linked to another directory on another partition.
• Disable SELinux
  • This was done as it was not needed for the tests, and also further options to make things work when docker selinux enforcement was on, was not investigated
  • So, if we can keep SELinux and still make things work later, changes to this text would be helpful
• docker was enabled to run as a service on restarts
  • Again this is optional, if dealing with an assigned machine, where everything has to be setup from scratch anyways

Prepare base images and networks

#docker pull centos
OR
#docker pull centos:7

NOTE: We work with centos in the examples below. As of this writing centos is the same as centos:7

#docker network create --gateway=172.20.0.1 --subnet=172.20.0.0/16 glfstest20

Explanation of the above command:

• create a new bridged network for use, names glfstest20
• –subnet for this network
• –gateway for this network

NOTE: This is not essential, but working on a separate bridged network can help isolating other containers using the default bridge, further this bridge can be configured with DNS in the future (see notes on editing /etc/hosts file)

Gluster brick container:

Create the gluster brick container

#docker run --name=centos-glfs-server1 --hostname=centos-glfs-server1 --volume=/d/docker-binds/glfs-node-1:/d: -dti --net glfstest20 --privileged=true centos

Explanation of the above command:

• –name is used as it is helpful in dealing with determining which container we are working with in docker container ps output
• –hostname is used as this is leveraged when creating volumes, and mounting them, instead of relying on static IPs in this configuration
• –volume is used to preserve the bricks across container restarts, this helps in case we expand the test cases to include things like upgrades, or down server cases
  • NOTE: /d/docker-binds/glfs-node-1 is a local path on the container host, bind mounted into the container as /d (just so we can use our .t files in the future that depends on /d/backends to be present)
• -dti creates the container detached (runs in the background), preserves the TTY and also STDIN
• –net uses the bridged network that we created
• –priveleged is used to be able to read/write xattrs to the bind mounted file system, as gluster bricks need this and a volume creation will fail if xattr support is not present on the brick directories

Enter the container

#docker exec -it $(docker ps -qf "name=centos-glfs-server1") bash

Explanation of the above command:

• docker ps command gets us the container ID of the container that we just created
• docker exec … bash, gets us into a bash shell on that container to run further commands
  THOUGHT: When we get into automation, we possibly will not use -it or run bash, but can pass in a bash script to run the rest of the commands

Install the gluster bits that need qualification

#yum -y update
#yum -y install centos-release-gluster310
#yum -y --enablerepo=centos-gluster310-test install glusterfs-server-3.10.5-1.el7

NOTE: centos-gluster310-test and glusterfs-server-3.10.5-1.el7 can be considered as the examples in this case, and should change in the future to whichever bits need testing.

The simple test case!

#glusterd
#gluster v create patchy replica 3 centos-glfs-server1:/d/brick1 centos-glfs-server1:/d/brick2 centos-glfs-server1:/d/brick3 centos-glfs-server1:/d/brick4 centos-glfs-server1:/d/brick5 centos-glfs-server1:/d/brick6 force
#gluster volume start patchy

Above series of commands should mostly be self explanatory for Gluster users. Of course, if we are to run different test cases, then those would take over at this point and need coordination between the client and server containers, but this is a simple setup as of now.

Cleaning up the server container

These are done post the client container is prepared and the tests are completed, but left here as they are a part of the server/brick container workflow.

** exit the bash shell (if you were in it)
- docker stop $(docker ps -qf "name=centos-glfs-server1")
- docker rm $(docker ps -qaf "name=centos-glfs-server1")
- (cleanup host persistent mount space)
    - rm -rf /d/docker-binds/glfs-node-1/*

Gluster client container:

#docker run --name=centos-glfs-client1 --hostname=centos-glfs-client1 --cap-add SYS_ADMIN --device /dev/fuse -dti --net glfstest20 centos

Explanation of the above command:

• options used for brick/server container is not explained here
• –cap-add we add this capability as we are using gluster FUSE mount within the client container for our testing

#docker exec -it $(docker ps -qf "name=centos-glfs-client1") bash
#yum -y update
#yum -y install centos-release-gluster310

We first install glusterfs, as we have 3.11 versions of the same as well in the centos gluster310 test repositories, and that leads to a clash in packages.

#yum -y --enablerepo=centos-gluster310-test install glusterfs-3.10.5-1.el7

#yum -y --enablerepo=centos-gluster310-test install glusterfs-fuse-3.10.5-1.el7
#yum -y install fuse fuse-libs
#mount -t glusterfs centos-glfs-server1:patchy /mnt
*** run some creates and tests as needed!
*** exit the container bash shell
#docker stop $(docker ps -qf "name=centos-glfs-client1")
#docker rm $(docker ps -qaf "name=centos-glfs-client1")

Ideally now would be a good time to cleanup the server container as well.