# Running gluster regression tests within a container

This document details how to run regression tests within a container, and more importantly useful when attempting to run the same within different distribution flavors, as machines with these flavors may not be easily available.

**NOTE:** All tests do not work within a container with the steps below, there are know hangs that happen in the following tests that are yet to be debugged and resloved. In other words **"your mileage may vary!"**

Tests that hang (you have been **warned!**):
- ./tests/basic/quota-nfs.t
- ./tests/bugs/nfs/bug-1161092-nfs-acls.t

## Setup docker
On a fresh Fedora (26 is what was used), run the following to setup docker:

#dnf install docker
#mkdir /d/docker
#rmdir /var/lib/docker/
#ln -s /d/docker /var/lib/docker
#vi /etc/sysconfig/docker
    - Remove selinux enforcement for things to work right!
#systemctl enable docker
#systemctl start docker
#docker run hello-world

The above does a few specific things that may not be needed for other installations, these (and reasons for the same) are detailed below,
- /var/lib/docker is soft linked to a different directory (in the case above /d/docker)
- This is done, so that all docker images and work do not use up space in the root partition (which is where /var/lib was in this case), and hence before even starting docker, this was soft linked to another directory on another partition.
- Disable SELinux
- This was done as it was not needed for the tests, and also further options to make things work when docker selinux enforcement was on, was not investigated
- So, if we can keep SELinux and still make things work later, changes to this text would be helpful
- docker was enabled to run as a service on restarts
- Again this is optional, if dealing with an assigned machine, where everything has to be setup from scratch anyways

## Prepare base images and networks

**NOTE:** CentOS7 is used as an example below, for other flavors, use the appropriate base image

#docker pull centos
OR
#docker pull centos:7

**NOTE:** We work with centos in the examples below. As of this writing centos is the same as centos:7

#docker network create --gateway=172.20.0.1 --subnet=172.20.0.0/16 glfstest20

Explanation of the above command:
- create a new bridged network for use, names glfstest20
- --subnet for this network
- --gateway for this network

**NOTE:** This is not essential, but working on a separate bridged network can help isolating other containers using the default bridge, further this bridge can be configured with DNS in the future (see notes on editing /etc/hosts file)

## Preparing the container

**NOTE:** CentOS7 is used as an example below, for other flavors, use the appropriate base image

### Getting the base container ready for use

#docker run --name=centos-glfs-server1 --hostname=centos-glfs-server1 --volume=/dev:/dev --volume=/d/docker-binds/glfs-node-1:/d: --volume=/root/glusterfs:/root/glusterfs: --volume=/root/build:/root/build: -dti --net glfstest20 --privileged=true --cpuset-cpus 3 centos

Explanation of the above command:
- --name is used as it is helpful in dealing with determining which container we are working with in `docker container ps` output
- --hostname is used as this is leveraged when creating volumes, and mounting them, instead of relying on static IPs in this configuration
- --volume is used to preserve the bricks across container restarts, this helps in case we expand the test cases to include things like upgrades, or down server cases
- **NOTE:** /d/docker-binds/glfs-node-1 is a local path on the container host, bind mounted into the container as /d (just so we can use our .t files in the future that depends on /d/backends to be present)
- -dti creates the container detached (runs in the background), preserves the TTY and also STDIN
- --net uses the bridged network that we created
- --priveleged is used to be able to read/write xattrs to the bind mounted file system, as gluster bricks need this and a volume creation will fail if xattr support is not present on the brick directories
- cpuset-cpus is used to see how best things perform on low CPU counts
- /dev is bind mounted to give tests ability to create the loop devices (for snapshot related tests in particular, possibly among others)
- /root/glusterfs is the git clone on the host that is again bind mounted within the container to enable compile..install..test

### Prepare LVM to work within the container

### Run it! and enter the container

#docker exec -it $(docker ps -qf "name=centos-glfs-server1") bash

Explanation of the above command:
- docker ps command gets us the container ID of the container that we just created
- docker exec ... bash, gets us into a bash shell on that container to run further commands
**THOUGHT:** When we get into automation, we possibly will not use -it or run bash, but can pass in a bash script to run the rest of the commands

### Steps within the container shell

### Compile, install gluster

NOTE: We build inside a different directory, helps throw the build away once we are done with the container, and keeps code directory cleaner

### Get relevant services started

<more?>

### Run the test of choice (or all)

## Cleaning up the container
These are done post the client container is prepared and the tests are completed, but left here as they are a part of the server/brick container workflow.

** exit the bash shell (if you were in it)
- docker stop $(docker ps -qf "name=centos-glfs-server1")
- docker rm $(docker ps -qaf "name=centos-glfs-server1")
- (cleanup host persistent mount space)
    - rm -rf /d/docker-binds/glfs-node-1/*