# IsardVDI Master/Slave cluster Ubuntu 22.04
## Base Ubuntu 22.04 packages
### Enabling Backports Repository
Copy and paste these lines of code into your **/etc/apt/sources.list** if only it doesn't have these lines. Please remember that lines beginning with ## or # are just comments and ignored by computer.
```
deb http://archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse
sudo apt-get update
```
```
sudo apt-get install -t jammy-backports iptables
apt install -y rsync ipmitool wget screen iperf ethtool tshark
apt install -y network-manager lvm2
apt install -y gnupg2 mdadm
apt install -y python3-natsort
apt install -y corosync pcs gnutls-bin
```
### Upgrade resource-agents
```bash
apt install resource-agents -t jammy-backports
```
### Change Timedatectl
```
timedatectl set-timezone Europe/Madrid
```
### Docker & docker-compose
Instalar docker-ce
https://docs.docker.com/engine/install/ubuntu/
```
sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update
sudo apt-get install \
ca-certificates \
curl \
gnupg \
lsb-release
sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
#test
docker run hello-world
```
Install docker-compose
```
apt install -y python3-pip
pip3 install docker-compose
```
Upgrade docker compose v2
```
# create the docker plugins directory if it doesn't exist yet
mkdir -p ~/.docker/cli-plugins
# download the CLI into the plugins directory
curl -sSL https://github.com/docker/compose/releases/download/v2.0.1/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose
# make the CLI executable
chmod +x ~/.docker/cli-plugins/docker-compose
```
### Disable apparmor
```
sudo apt remove --assume-yes apparmor
systemctl stop apparmor
systemctl disable apparmor
systemctl status apparmor
```
**Le decimos que queremos que network-manager maneje tanto las conexiones de ethernet como las de wireguard:**
```
mv /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf_orig
touch /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf
echo "[keyfile]" > /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf
echo "unmanaged-devices=*,except:type:wifi,except:type:ethernet,except:type:wireguard" >> /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf
```
**también hay que modificar la resolución dns para que la haga sin usar systemd**
```
mv /usr/lib/NetworkManager/conf.d/10-dns-resolved.conf /usr/lib/NetworkManager/conf.d/10-dns-resolved.conf:orig
echo "[main]" > /usr/lib/NetworkManager/conf.d/10-dns-resolved.conf
echo "dns=default" >> /usr/lib/NetworkManager/conf.d/10-dns-resolved.conf
```
Reiniciamos network-manager:
```
systemctl restart NetworkManager
```
### Install DRBD
```
apt update
wget https://packages.linbit.com/proxmox/dists/proxmox-7/drbd-9/pool/drbd-utils_9.23.1-1_amd64.deb
dpkg -i drbd-utils_9.23.1-1_amd64.deb
wget https://packages.linbit.com/proxmox/dists/proxmox-7/drbd-9/pool/drbd-dkms_9.2.2-1_all.deb
dpkg -i drbd-dkms_9.2.2-1_all.deb
dkms status
```
***ERROR apt update proxmox:***
Open Terminal (if it's not already open)
List existing keys:
```
$ sudo apt-key list
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
/etc/apt/trusted.gpg
--------------------
pub rsa4096 2017-05-08 [SCEA]
1EDD E2CD FC02 5D17 F6DA 9EC0 ADAE 6AD2 8A8F 901A
uid [ unknown] Sublime HQ Pty Ltd <support@sublimetext.com>
sub rsa4096 2017-05-08 [S]
pub rsa2048 2015-10-28 [SC]
BC52 8686 B50D 79E3 39D3 721C EB3E 94AD BE12 29CF
uid [ unknown] Microsoft (Release signing) <gpgsecurity@microsoft.com>
```
From here, we can export a key:
```
sudo apt-key export BE1229CF | sudo gpg --dearmour -o /usr/share/keyrings/microsoft.gpg
```
Note: The BE1229CF value comes from the last 8 characters of the pub code.
The following message will likely appear:
```
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
```
Now we can update our apt source file for the repository (e.g., /etc/apt/sources.list.d/microsoft.list), adding a signed-by tag:
```
deb [arch=amd64 signed-by=/usr/share/keyrings/microsoft.gpg] https://packages.microsoft.com/repos/edge/ stable main
```
Remove the original signature:
```
sudo apt-key del BE1229CF
```
## DRBD configuration
All aspects of DRBD are controlled in its configuration file, /etc/drbd.conf. Normally, this configuration file is just a skeleton with the following contents:
```
cat /etc/drbd.conf
```
By convention, /etc/drbd.d/global_common.conf contains the global and common sections of the DRBD configuration, whereas the .res files contain one resource section each.
## Sample DRBD Configuration File
You will get a sample DRBD configuration file under /usr/share/doc/packages/drbd. I will attach a copy of this drbd configuration file for reference
## Create DRBD resource
- A per-resource configuration file is usually named /etc/drbd.d/<resource>.res.
- Any DRBD resource you define must be named by specifying a resource name in the configuration.
- In this DRBD Tutorial we will name our resource as drbd1
- Every resource configuration must also have at least two on host sub-sections, one for every cluster node.
- All other configuration settings are either inherited from the common section (if it exists), or derived from DRBD’s default settings.
```
vim /etc/drbd.d/drbd1.res
```
Configuration file (drbd.9.X):
```
resource drbd1 {
volume 0 {
meta-disk internal;
device minor 1;
disk "/dev/sdb1";
}
volume 1 {
meta-disk internal;
device minor 2;
disk "/dev/sdb2";
}
on testing-drbd-1 { #<-- hostname must match `uname -n` output
node-id 0; #<-- Assign a unique node-id
address 10.100.1.230:7789; #<-- IP Address to be used to connect to the node with port
}
on testing-drbd-2 {
node-id 1;
address 10.100.1.233:7789;
}
connection-mesh {
hosts testing-drbd-1 testing-drbd-2;
}
}
```
> Note: that the /var/lib/drbd directory is needed beforehand. If it was not created previously when you installed DRBD, create it manually before proceeding
```
mkdir /var/lib/drbd
```
- When we installed DRBD earlier, a utility called drbdadm was installed as well
- drbdadm is intended to be used for the administration of DRBD resources, such as our newly configured volume
- The first step of DRBD Tutorial and Linux Disk Replication in starting and bringing a DRBD resource online is to initialize its metadata
Now using the drbdadm utility initialize the meta data storage. On each server execute:
```
sudo drbdadm create-md drbd1
sudo drbdadm up drbd1
```
Next, on both hosts, start the drbd daemon:
```
sudo systemctl start drbd.service
```
On the drbd01, or whichever host you wish to be the primary, enter the following:
```
drbdadm primary --force drbd1
```
### Pacemaker
```
apt -y install pacemaker pcs resource-agents
systemctl enable pacemaker
systemctl enable corosync
systemctl enable pcsd
```
```
# set cluster admin password
passwd hacluster
pcs cluster destroy
# authorize among nodes
pcs host auth nodo1 addr=10.10.20.11 nodo2 addr=10.10.20.12 nodo3 addr=10.10.20.13 -u hacluster
# configure cluster
pcs cluster setup ha_cluster nodo1 addr=10.10.20.11 nodo2 addr=10.10.20.12 nodo3 addr=10.10.20.13 (--force)
```
```sh
# start services for cluster
pcs cluster start --all
# set auto-start
pcs cluster enable --all
(systemctl enable --now pcsd)
```
Check that everything looks right:
```
pcs status
```
LA PARTE DEL DRBD EN PACEMAKER NO FUNCIONABA POR LA VERSIÓN DE DRBD KERNEL Y NECESITÁBAMOS : https://github.com/LINBIT/drbd-utils/blob/4adf320f741e824b89737ef7d996db27afbd27d5/scripts/drbd.ocf#L1092 meter este ocf en /usr/lib/ocf/resource.d/heartbeat/, darle permisos 755 y volver a montar el DRBD si ya lo habíamos montado.
-----------------------------------------
### Firewalld & Fail2ban
#### Install firewalld and set iptables backend instead of nftables
```bash
apt install firewalld -y
# Setting iptables to not use nf_tables
update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
# Setting firewalld to use iptables
sed -i 's/FirewallBackend=nftables/FirewallBackend=iptables/g' /etc/firewalld/firewalld.conf
```
#### Install fail2ban
```bash
apt install -y fail2ban
```
#### Set up firewalld
Add **zone infrastructure** with unrestricted access
```bash
firewall-cmd --permanent --new-zone=infrastructure
# Source allowed (all internal networks)
firewall-cmd --permanent --zone=infrastructure --add-source=172.31.0.0/21
firewall-cmd --permanent --zone=infrastructure --add-rich-rule='rule family="ipv4" source address="172.31.0.0/21" accept'
```
Add **zone hyper** with access to 2022 and video ports 5900-6899
```bash
firewall-cmd --permanent --new-zone=hyper
# Sources allowed
firewall-cmd --permanent --zone=hyper --add-source=172.16.254.200/32
firewall-cmd --permanent --zone=hyper --add-source=172.16.254.201/32
# Hypervisor ssh port from isard-engine
firewall-cmd --permanent --zone=hyper --add-rich-rule='rule family="ipv4" source address="172.16.254.200/32" port port="2022" protocol="tcp" accept' --permanent
firewall-cmd --permanent --zone=hyper --add-rich-rule='rule family="ipv4" source address="172.16.254.201/32" port port="2022" protocol="tcp" accept' --permanent
# Forward this port to isard-hypervisor internal IP
firewall-cmd --permanent --zone=hyper --add-forward-port=port=2022:proto=tcp:toport=22:toaddr=172.18.255.17 --permanent
# Video ports without proxy
firewall-cmd --permanent --zone=hyper --add-rich-rule='rule family="ipv4" source address="172.16.254.200/32" port port="5900-6899" protocol="tcp" accept' --permanent
firewall-cmd --permanent --zone=hyper --add-rich-rule='rule family="ipv4" source address="172.16.254.201/32" port port="5900-6899" protocol="tcp" accept' --permanent
# Forward those ports to isard-hypervisor internal IP
firewall-cmd --permanent --zone=hyper --add-forward-port=port=5900-6899:proto=tcp:toport=5900-6899:toaddr=172.18.255.17 --permanent
```
#### LETS RESTART EVERYTHING
```
systemctl restart firewalld
systemctl stop docker
systemctl start docker
systemctl restart fail2ban
```
# Impi
- Node1 172.31.4.201
- Node2 172.31.4.202
- Node2 172.31.4.203
```
ipmitool lan print 1
ipmitool lan set 1 ipsrc static
ipmitool lan set 1 ipaddr 172.31.4.21
ipmitool lan set 1 netmask 255.255.255.0
ipmitool lan set 1 defgw ipaddr 172.31.4.254
ipmitool lan set 1 ipsrc static
ipmitool lan set 1 ipaddr 172.31.4.22
ipmitool lan set 1 netmask 255.255.255.0
ipmitool lan set 1 defgw ipaddr 172.31.4.254
#ipmitool lan set 1 vlan id 104
ipmitool user set name 3 pacemaker
ipmitool channel setaccess 1 3 link=on ipmi=on callin=on privilege=3
# Download Supermicro IPMICFG: https://www.supermicro.com/SwDownload/SwSelect_Free.aspx?cat=IPMI
0penSourceRocks
./IPMICFG-Linux.x86_64 -user setpwd 3 <passwd>
ipmitool user enable
```
IMPORTANT: You SHOULD have some kind of STONITH to have a working cluster without extrange behaviours!!!
Check that you have access to your IPMI on both servers:
```
root@cluster1:~# ipmitool -I lanplus -H 172.31.4.21 -U pacemaker -P 0penSourceRocks chassis status
System Power : on
Power Overload : false
Power Interlock : inactive
Main Power Fault : false
Power Control Fault : false
Power Restore Policy : previous
Last Power Event :
Chassis Intrusion : inactive
Front-Panel Lockout : inactive
Drive Fault : false
Cooling/Fan Fault : false
```
And that you are able to request status with pacemaker stonith resource:
```console
root@cluster1:~# fence_ilo3 -a nodo2-ipmi -l ADMIN -p XXXXXXX -o status
Status: ON
```
```
gw_out='192.168.0.254'
dns_out='8.8.8.8'
```
# NET Config
/etc/hosts
```
172.31.1.11 nodo1-drbd nodo1.UniCluster # 10G (A)
172.31.3.11 nodo1-nfs # 10G (B)
172.31.4.11 nodo1-stonith nodo1 # 1G (A)
192.168.0.11 nodo1-out # 1G (B)
172.31.2.11 nodo1-infraip # 1G (C)
# # 1G VLANS TRUNK
172.31.4.21 nodo1-ipmi # SHOULD BE VISIBLE FROM nodo1-stonith
172.31.1.12 nodo2-drbd nodo2.UniCluster
172.31.3.12 nodo2-nfs
172.31.4.12 nodo2-stonith nodo2
192.168.0.12 nodo2-out
172.31.2.12 nodo2-infraip
#
172.31.4.22 nodo2-ipmi
172.31.1.13 nodo3-drbd nodo3.UniCluster
172.31.3.13 nodo3-nfs
172.31.4.13 nodo3-stonith nodo3
192.168.0.13 nodo3-out
172.31.2.13 nodo3-infraip
#
172.31.4.23 nodo3-ipmi
```
Floating IPs:
- nfs: 172.31.3.10
- isard infraip: 172.31.2.10 (Used for other hypervisors to know where it is main isard)
- isard publicip: 192.168.0.10 (will move to other nodes with primary drbd and main isard)
### Set interfaces
host=11 # change for other nodes!
#### SET DRBD INTERFACE
##### DRBD intra cluster disk block sync
##### ISOLATED 10G
##### Ex: nodo1-drbd
```
interface='eno1np0'
network='172.31.1'
MAC=$(cat /sys/class/net/$interface/address)
echo 'SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="'$MAC'", ATTR{mtu}="9000", KERNEL=="e*", NAME="drbd"' >> /etc/udev/rules.d/70-net-persistent.rules
ip link set $interface down
ip link set $interface name drbd
ip link set drbd mtu 9000
ip link set drbd up
nodo1-ipm
nmcli con add con-name drbd ifname drbd type ethernet ip4 $network.$host/24 802-3-ethernet.mtu 9000 connection.autoconnect yes
nmcli con up drbd
```
##### NFS
##### ISOLATED 10G
##### Ex: nodo1-nfs
```
interface='eno2np1'
network='172.31.3'
MAC=$(cat /sys/class/net/$interface/address)
echo 'SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="'$MAC'", ATTR{mtu}="9000", KERNEL=="e*", NAME="nfs"' >> /etc/udev/rules.d/70-net-persistent.rules
ip link set $interface down
ip link set $interface name nfs
ip link set nfs mtu 9000
ip link set nfs up
nmcli con add con-name nfs ifname nfs type ethernet ip4 $network.$host/24 802-3-ethernet.mtu 9000 connection.autoconnect yes
nmcli con up nfs
```
NOTE: Check that you can connect from both clústers at 10G. If you don't get near 10G check MTU9000 on this ports at switch.
node1: iperf -s
node2: iperf -c 172.31.1.11
##### CLUSTER
##### 1G WITH IPMI
##### Ex: nodo1
```
if_name=enp197s0f3
if_newname=cluster
ipaddr=172.31.4.13/24
MAC=$(cat /sys/class/net/$if_name/address)
echo 'SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="'$MAC'", KERNEL=="e*", NAME="'$if_newname'"' >> /etc/udev/rules.d/70-net-persistent.rules
ip link set $if_name down
ip link set $if_name name $if_newname
ip link set $if_newname up
nmcli con add con-name $if_newname ifname $if_newname type ethernet
nmcli con mod $if_newname ipv4.method manual ipv4.addresses $ipaddr
nmcli con mod $if_newname ipv6.method ignore
nmcli con up $if_newname
```
##### OUT
##### 1G
##### Ex: nodo1-out
```
if_name=enp197s0f1
if_newname=out
ipaddr=192.168.0.1/24
ipgw=192.168.0.254
ipdns=10.15.27.1,8.8.8.8
MAC=$(cat /sys/class/net/$if_name/address)
echo 'SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="'$MAC'", KERNEL=="e*", NAME="'$if_newname'"' >> /etc/udev/rules.d/70-net-persistent.rules
ip link set $if_name down
ip link set $if_name name $if_newname
ip link set $if_newname up
nmcli con add con-name $if_newname ifname $if_newname type ethernet
nmcli con mod $if_newname ipv4.method manual ipv4.addresses $ipaddr ipv4.gateway $ipgw ipv4.dns $ipdns
nmcli con mod $if_newname ipv6.method ignore
nmcli con up $if_newname
```
##### INFRAIP
##### 1G
##### Ex: nodo1-infraip
```
if_name=enp197s0f2
if_newname=infraip
ipaddr=172.31.2.13/24
MAC=$(cat /sys/class/net/$if_name/address)
echo 'SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="'$MAC'", KERNEL=="e*", NAME="'$if_newname'"' >> /etc/udev/rules.d/70-net-persistent.rules
ip link set $if_name down
ip link set $if_name name $if_newname
ip link set $if_newname up
nmcli con add con-name $if_newname ifname $if_newname type ethernet
nmcli con mod $if_newname ipv4.method manual ipv4.addresses $ipaddr
nmcli con mod $if_newname ipv6.method ignore
nmcli con up $if_newname
```
##### VLANS TRUNK
##### 1G
```
ip link set enp197s0f3 up
```
Check for vlans seen in trunk:
```
tshark -a duration:260 -i vlans -Y "vlan" -x -V 2>&1 |grep -o " = ID: .*" |awk '{ print $NF }'
```
# CLUSTER
## Set up pacemaker cluster
ssh-keygen
```
passwd hacluster
systemctl enable pacemaker
systemctl enable corosync
systemctl enable pcsd
```
## Authenticate cluster servers
En un nodo
```
pcs cluster destroy
pcs host auth nodo1 nodo2 nodo3 -u hacluster
```
## Set the new cluster and start it
```sh
pcs cluster setup UniCluster nodo1 addr=172.31.4.11 nodo2 addr=172.31.4.12 nodo3 addr=172.31.4.13
pcs cluster start
pcs cluster start nodo2
pcs cluster start nodo3
(systemctl enable --now pcsd)
```
Check that everything looks right:
```
pcs status
```
## Stonith IPMI resource
```
pcs stonith create stonith1 fence_ipmilan pcmk_host_list="nodo1" ipaddr="nodo1-ipmi" login=ADMIN passwd=0penSourceRocks lanplus=true power_wait=4 op monitor interval=60s
pcs stonith create stonith2 fence_ipmilan pcmk_host_list="nodo2" ipaddr="nodo2-ipmi" login=ADMIN passwd=0penSourceRocks lanplus=true power_wait=4 op monitor interval=60s
pcs stonith create stonith3 fence_ipmilan pcmk_host_list="nodo3" ipaddr="nodo3-ipmi" login=ADMIN passwd=0penSourceRocks lanplus=true power_wait=4 op monitor interval=60s
pcs constraint location stonith1 avoids nodo1=INFINITY
pcs constraint location stonith2 avoids nodo2=INFINITY
pcs constraint location stonith3 avoids nodo3=INFINITY
```
```
pcs cluster stop --all
pcs quorum update wait_for_all=0
pcs cluster start --all
pcs property set no-quorum-policy=ignore
#pcs property set startup-fencing=false
pcs property set stonith-enabled=true
```
# DRBD
Copy drbd config files (check that disks are yours!!)
```
drbdadm create-md UniCluster
drbdadm adjust all
drbdadm up UniCluster
# On one node to start conection and sync
drbdadm -- --discard-my-data connect UniCluster
drbdadm primary UniCluster --force
```
# DRBD Master/Slave resource
```
pcs resource create drbd ocf:linbit:drbd drbd_resource=UniCluster op monitor interval=10s
pcs resource promotable drbd \
promoted-max=1 promoted-node-max=1 clone-max=3 clone-node-max=1 \
notify=true
```
# LVM
## create lvm cluster
```
/etc/lvm/lvm.conf
filter = [ "r|/dev/nvme*|" ]
volume_list = [ ]
```
```
pvcreate /dev/drbd1001
pvcreate /dev/drbd1002
vgcreate vgUniCluster /dev/drbd1001 /dev/drbd1002
lvcreate -l 100%FREE -n lvUniCluster vgUnicluster
mkfs.xfs /dev/vgUniCluster/UniCluster
```
### UniCluster RESOURCES
```
pcs resource create lvm LVM volgrpname=vgUniCluster exclusive=true \
--group storage
```
# XFS filesystem
```
pcs resource create mountfs Filesystem \
device=/dev/vgUniCluster/lvUniCluster directory=/opt/isard \
fstype=xfs "options=defaults,noatime,nodiratime,noquota" op monitor interval=10s \
--group storage
pcs constraint order promote drbd-clone then start storage symmetrical=true kind=Mandatory
pcs constraint colocation add storage with master drbd-clone
```
# NFS SERVER
## NFS SERVER CONFIG v4
/etc/default/nfs-common
```
NEED_STATD="no"
NEED_IDMAPD="yes"
```
/etc/default/nfs-kernel-server
```
RPCNFSDARGS="--lease-time 10 --grace-time 10"
RPCNFSDOPTS="-N 2 -N 3"
RPCMOUNTDOPTS="--manage-gids -N 2 -N 3 -H 172.31.3.10"
```
```
systemctl mask --now rpc-statd.service rpcbind.service rpcbind.socket
```
```
cp etcs/heartbeat/* /usr/lib/ocf/resource.d/heartbeat/
chmod 755 /usr/lib/ocf/resource.d/heartbeat/compose
chmod 755 /usr/lib/ocf/resource.d/heartbeat/nfsserver4
```
## Pacemaker NFS SERVER
```
pcs resource create nfs_ip IPaddr2 \
ip=172.31.3.10 cidr_netmask=24 --group UniCluster
```
```
mkdir -p /opt/isard/nfsinfo
pcs resource create nfs-daemon nfsserver4 \
nfs_shared_infodir=/opt/isard/nfsinfo nfs_ip=172.31.3.10 \
--group UniCluster
```
```
pcs resource create nfs-root exportfs \
clientspec=172.31.3.0/255.255.255.0 \
options=rw,sync,no_root_squash \
directory=/opt \
fsid=0 --group UniCluster
```
```
pcs resource create nfs-isard exportfs \
clientspec=172.31.3.0/255.255.255.0 \
options=rw,sync,no_root_squash directory=/opt/isard \
fsid=1 \
wait_for_leasetime_on_stop=true \
--group UniCluster
```
## NFS CLIENT
```
pcs resource create nfs-client Filesystem \
device=172.31.3.10:/isard directory="/opt/isard" \
fstype="nfs" "options=defaults,noatime" op monitor interval=10s
pcs resource clone nfs-client clone-max=8 clone-node-max=1 notify=true
```
```
pcs constraint order UniCluster then nfs-client-clone \
# symmetrical=false
pcs constraint colocation add nfs-client-clone with UniCluster -INFINITY
```
# DOCKER COMPOSE
```
pcs resource create isard_infraip IPaddr2 \
ip=172.31.2.10 cidr_netmask=24 --group isard-main
pcs resource create isard_publicip IPaddr2 \
ip=192.168.0.10 cidr_netmask=24 --group isard-main
```
Now clone repository in /opt/isard/src/isard folder for main and .../hyperX for each hyper
```
cd /opt/isard/src (isard & isard-hypX)
git clone https://gitlab.com/isard/isardvdi isard
git clone https://gitlab.com/isard/isardvdi hyper1
git clone https://gitlab.com/isard/isardvdi hyper2
...
```
Now edit isardvdi.cfg in each folder and set:
- HYPERVISOR_NUMBER to 0,1,2...
- VIDEO_PROXY_HOSTS=isard-hypervisor,<ip of each hypervisor for video>,<...>
```
./build.sh
docker-compose -f docker-compose.no-stats.yml pull
docker-compose -f docker-compose.hypervisor-standalone.yml pull
```
Now check that you can start and stop both without errors on all servers:
```
docker-compose -f docker-compose.no-stats.yml up -d (You should be able to get web interface)
docker-compose -f docker-compose.no-stats.yml down
docker-compose -f docker-compose.hypervisor-standalone.no-stats.yml up -d
docker-compose -f docker-compose.hypervisor-standalone.no-stats.yml down
```
```
systemctl disable --now nfs-kernel-server
systemctl disable --now nfs-server
```
NOTE: Activate fencing now!
```
pcs property set stonith-enabled=true
```
NOTE: Activate cluster monitoring rr (corosync.conf)
Refer to example corosync.conf
Isard main docker-compose
```
pcs resource create isard ocf:heartbeat:compose \
conf="/opt/isard/src/isard/docker-compose.no-stats.yml" \
env_path="/opt/isard/src/isard" \
force_kill="true" \
op start interval=0s timeout=300s \
op stop interval=0 timeout=300s \
op monitor interval=60s timeout=60s \
--group isard-main
```
Avoid starting isard-main without storage and not in another node without storage
```
pcs constraint order start storage then start isard-main
pcs constraint colocation add isard-main with storage
```
Isard hypervisors
```
pcs resource create hyper1 ocf:heartbeat:compose \
conf="/opt/isard/src/hyper1/docker-compose.hypervisor-standalone.no-stats.yml" \
env_path="/opt/isard/src/hyper1" \
force_kill="true" \
op start interval=0s timeout=300s \
op stop interval=0 timeout=300s \
op monitor interval=60s timeout=60s
pcs resource create hyper2 ocf:heartbeat:compose \
conf="/opt/isard/src/hyper2/docker-compose.hypervisor-standalone.no-stats.yml" \
env_path="/opt/isard/src/hyper2" \
force_kill="true" \
op start interval=0s timeout=300s \
op stop interval=0 timeout=300s \
op monitor interval=60s timeout=60s
```
Constraints to avoid running in same host
```
pcs constraint order nfs-client-clone then hyper1
pcs constraint colocation add hyper1 with isard-main -INFINITY
pcs constraint order nfs-client-clone then hyper2
pcs constraint colocation add hyper2 with isard-main -INFINITY
pcs constraint colocation add hyper1 with hyper2 -INFINITY
```
This is needed when less than three nodes in the cluster to avoid node waiting forever to get quorum
```
pcs quorum update wait_for_all=0
```
This will avoid resources moving unless the node fails
```
pcs resource defaults resource-stickiness=100
```
Sign in with Wallet
Connect another wallet