# Setup CentOS/Red Hat 8 HPC Laboratory Environment

This document describes how I setup my HPC laboratory environment.

1. Install OS with "Server" preset.
2. Setup hostname and network.

# Easy way
nmtui

# Hard way
# Fill appropriate values in [...]
hostnamectl set-hostname [...]
nmcli connection show
nmcli connection modify [...] ipv4.method manual
nmcli connection modify [...] ipv4.addresses [...]
nmcli connection modify [...] ipv4.gateway [...]
nmcli connection modify [...] ipv4.dns "1.1.1.1,8.8.8.8,9.9.9.9"
nmcli connection modify [...] connection.autoconnect yes

systemctl restart NetworkManager

3. Setup NTP client.

vim /etc/chrony.conf
# Specify time servers
# ----------
server clock.stdtime.gov.tw
server tick.stdtime.gov.tw
server time.stdtime.gov.tw
server tock.stdtime.gov.tw
server watch.stdtime.gov.tw
# ----------

systemctl start chronyd
systemctl enable chronyd

4. Update and install packages.

dnf makecache
dnf update -y

dnf module install -y python27 python38
# dnf install -y "@python27" "@python38"

dnf group install -y "Development Tools"
# dnf install -y "@Development Tools"
dnf module install -y mercurial subversion
# dnf install -y "@mercurial" "@subversion"
dnf install -y gcc-gfortran
dnf install -y gcc-toolset-10
dnf install -y cmake ncompress tcsh tmux xorg-x11-xauth

5. Disable SELinux.

sed -e "s/SELINUX=enforcing/SELINUX=disabled/g" -i /etc/selinux/config

6. Disable firewall.

systemctl stop firewalld
systemctl disable firewalld

7. Install Mellanox OFED for Infiniband.

Download Mellanox OFED from [here](https://www.mellanox.com/products/infiniband-drivers/linux/mlnx_ofed).

# Install dependencies
dnf module install -y python36
# dnf install -y "@python36"
dnf install -y tcl tcsh tk

# Install Mellanox OFED with HPC preset
./mlnxofedinstall --hpc --without-fw-update

8. Install Singularity.

* Install `go` compiler.

# Download go compiler binary archive
wget https://golang.org/dl/go1.16.7.linux-amd64.tar.gz

# Extract go compiler binary archive
tar -xvf go1.16.7.linux-amd64.tar.gz -C /usr/local

# Add go compiler to PATH
echo 'export PATH="${PATH}:/usr/local/go/bin"' >>.bashrc
export PATH="${PATH}:/usr/local/go/bin"

# Check if go compiler works
go version

* Compile and install Singularity from source.

# Install dependencies
dnf install -y cryptsetup libseccomp-devel libuuid-devel openssl-devel squashfs-tools

export VERSION=3.8.1

# Download Singularity source archive
wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-ce-${VERSION}.tar.gz

# Extract Singularity source archive
tar -xvf singularity-ce-${VERSION}.tar.gz

# Configure Singularity
cd singularity-ce-${VERSION}
./mconfig

# Compile and install Singularity
cd builddir
make
make install

# Check if Singularity works
singularity version

* Enable shell auto-completion for Singularity.

echo '. /usr/local/etc/bash_completion.d/singularity' >>.bashrc
. /usr/local/etc/bash_completion.d/singularity

9. Install Docker in rootless mode.

* Remove conflicting packages.

dnf remove -y buildah podman

* Install Docker.

# Add Docker repository
wget https://download.docker.com/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
dnf makecache

# Install dependencies
dnf install -y fuse-overlayfs iptables

# Install Docker packages
dnf install -y containerd.io docker-ce docker-ce-cli

# Start and enable Docker service
systemctl start docker
systemctl enable docker

# Check Docker status
docker info

* Configure Docker in rootless mode for a user.

**LOGIN AS THE SPECIFIC USER FIRST!**

# Configure Docker in rootless mode
dockerd-rootless-setuptool.sh install

# Set `DOCKER_HOST` environment variable
echo 'export DOCKER_HOST="unix://${XDG_RUNTIME_DIR}/docker.sock"' >>.bashrc
export DOCKER_HOST="unix://${XDG_RUNTIME_DIR}/docker.sock"

# Check Docker status
docker info

Note: There is an alternative way to let users interact with Docker by **adding them to the `docker` group**. However, this method in fact **grants privileges equivalent to the `root` user** so it is unsuitable for use in an HPC environment.