Install Cuju & libvirt on CentOS7

# Install Cuju & libvirt on CentOS7 [Install Cuju & libvirt on Ubuntu18](https://hackmd.io/@4jocGoC0SRi106skGiCKfw/rkL4V50kO) Here is the architecture diagram for Cuju, you will need three node: **primary node**, **backup node** and **NFS server** to build a complete system. ![](https://i.imgur.com/NE99Bsd.png) ## NFS server step 1. Install the NFS service (Network FileSystem) in NFS node ```shell $ sudo yum install nfs-utils ``` 2. Create nfsfolder ```shell $ mkdir /home/[your username]/nfsfolder ``` 3. Insert this line in ==/etc/exports== to add your NFS folder: ```shell /home/[your username]/nfsfolder *(rw,no_root_squash,no_subtree_check) ``` 4. After editing /etc/exports, setting **selinux** and **firewall** ```shell $ sudo setsebool -P nfs_export_all_rw on $ sudo setsebool -P nfs_export_all_ro on $ sudo firewall-cmd --permanent --zone=public --add-service=nfs $ sudo firewall-cmd --reload ``` 5. Restart nfs-server ```shell $ sudo systemctl restart nfs-server $ sudo systemctl enable nfs-server ``` 6. Put the VM image in this NFS, you can creat image by youself, or download our example image **[Ubuntu-16.04 VM image](https://drive.google.com/file/d/0B9au9R9FzSWKNjZpWUNlNDZLcEU/view)** - Or you can use the script below to get the image in terminal ```shell $ wget --load-cookies /tmp/download.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/download.txt --keep-session-cookies 'https://docs.google.com/uc?export=download&id=0B9au9R9FzSWKNjZpWUNlNDZLcEU' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=0B9au9R9FzSWKNjZpWUNlNDZLcEU" -O 'Ubuntu20G-1604.tar.gz' && rm -rf /tmp/download.txt $ tar -zxvpf Ubuntu20G-1604.tar.gz -C ~/nfsfolder ``` ## Primay and backup node ### Step 1 Install CentOS operating system #### Install 1. Download CentOS7.8.2003 iso, you can download from **[here](https://vault.centos.org/7.8.2003/isos/x86_64/)** 2. Install CentOS - **Graphical user interface is not selected in the preset installation options** - If you need, you can select **gnome desktop** during installation ![](https://i.imgur.com/ByOBxZJ.png) #### Download kernel's file 1. Download the compressed file **[kernel4_4_21_rpm.tar.gz](https://drive.google.com/uc?export=download&id=1TwAKCfz96PYdFKqga_sd0VfVQRpLij6P)** and put it in the home folder. You can use the script below to get the file in terminal ```shell $ wget --load-cookies /tmp/download.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/download.txt --keep-session-cookies 'https://docs.google.com/uc?export=download&id=1TwAKCfz96PYdFKqga_sd0VfVQRpLij6P' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1TwAKCfz96PYdFKqga_sd0VfVQRpLij6P" -O 'kernel4_4_21_rpm.tar.gz' && rm -rf /tmp/download.txt ``` 2. Decompress ==kernel4_4_21_rpm.tar.gz== ```shell $ tar -zxvpf kernel4_4_21_rpm.tar.gz -C ~/ ``` #### Kernel update Since Cuju is only supported in kernel 4.4, the kernel needs to be updated from 3.10 to 4.4. We have made the rpm installation package, you only need to download and install the rpm to update the kernl version. 1. Run ==install_kernel.sh== in the kernel folder ```shell $ cd ~/kernel4_4_21_rpm $ ./install_kernel.sh ``` ![](https://i.imgur.com/L4a1dkE.png) 2. Set kernel 4.4 as the default kernel version ```shell $ sudo grub2-set-default "CentOS Linux (4.4.21) 7 (Core)" ``` 3. Reboot ```shell $ sudo reboot ``` 4. After reboot, check whether the kernel is successfully update ```shell $ uname -r ``` ![](https://i.imgur.com/Ss2nqQG.png) After updating the kernel successfully, start to install Cuju and libvirt ### Step 2 Install Cuju 1. Update the package in the operating system and install vncviewer first ```shell $ sudo yum update $ sudo yum install tigervnc ``` 2. Download the compressed file **[cuju_libvirt_rpm.tar.gz](https://drive.google.com/uc?export=download&id=1C4U2DIJY5WZ_pEa_MLuOXzADaf8KoZX8)** and put it in the home folder. You can use the script below to get the file in terminal ```shell $ wget --load-cookies /tmp/download.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/download.txt --keep-session-cookies 'https://docs.google.com/uc?export=download&id=1C4U2DIJY5WZ_pEa_MLuOXzADaf8KoZX8' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1C4U2DIJY5WZ_pEa_MLuOXzADaf8KoZX8" -O 'cuju_libvirt_rpm.tar.gz' && rm -rf /tmp/download.txt ``` 3. Decompress ==cuju_libvirt_rpm.tar.gz== ```shell $ tar -zxvpf cuju_libvirt_rpm.tar.gz -C ~/ ``` 4. Run ==install_cuju.sh== in the cuju folder ```shell $ cd ~/cuju_libvirt_rpm/cuju $ ./install_cuju.sh ``` ![](https://i.imgur.com/WwKWfGH.png) ### Step 3 Install libvirt Because we have added some commands to libvirt, you need to remove the original libvirt and install our modified version. 1. Remove original libvirt ```shell $ sudo yum remove libvirt* ``` 2. Run ==install_libvirt.sh== in the libvirt folder ```shell $ cd ~/cuju_libvirt_rpm/libvirt $ ./install_libvirt.sh ``` ![](https://i.imgur.com/p80zpjm.png) 3. If the installation is complete, check whether libvirtd is successfully started ```shell $ sudo systemctl status libvirtd ``` ![](https://i.imgur.com/Jo9mAmZ.png) 4. In order to successfully start Cuju, the following environment settings are required, you can run ==setting.sh== in the libvirt folder to complete the setting. You need to add the user name allowed to use libvirt after the command ```shell $ cd libvirt $ ./setting.sh your-user-name ``` - The setting in the script - virt_use_nfs(line1): Set the selinux option, let virtual machine can use the image file in NFS - usermod(line2): Set the user group of the ssh user to libvirt. When connecting from another node, this user can use libvirt to perform operations on the VM - firewall-cmd(line3&4): Set the ports on the filewall to allow libvrit execute Live migration and Fault Tolerance ### Step 4 Cuju test environment setting ![](https://i.imgur.com/NE99Bsd.png) 1. Follow cuju's architecture to set up the bridge and network environment on **primary** and **backup** node, for example: - primary ```shell $ sudo vim /etc/sysconfig/network-scripts/ifcfg-br0 ``` ```shell DEVICE="br0" BOOTPROTO="static" IPADDR="192.168.123.1" NETMASK="255.255.255.0" GATEWAY="192.168.123.254" DNS1="8.8.8.8" ONBOOT="yes" TYPE="Bridge" NM_CONTROLLED="no" ``` ```shell $ sudo vim /etc/sysconfig/network-scripts/ifcfg-eth0 ``` ```shell TYPE="Ethernet" BOOTPROTO="none" DEVICE="eth0" ONBOOT="yes" NM_CONTROLLED=no BRIDGE=br0 ``` ```shell $ sudo vim /etc/sysconfig/network-scripts/ifcfg-eth1 ``` ```shell DEVICE="eth1" BOOTPROTO="static" IPADDR="192.168.122.11" NETMASK="255.255.255.0" ONBOOT="yes" TYPE="Ethernet" NM_CONTROLLED="no" ``` - backup ```shell $ sudo vim /etc/sysconfig/network-scripts/ifcfg-br0 ``` ```shell DEVICE="br0" BOOTPROTO="static" IPADDR="192.168.123.2" NETMASK="255.255.255.0" GATEWAY="192.168.123.254" DNS1="8.8.8.8" ONBOOT="yes" TYPE="Bridge" NM_CONTROLLED="no" ``` ```shell $ sudo vim /etc/sysconfig/network-scripts/ifcfg-eth0 ``` ```shell TYPE="Ethernet" BOOTPROTO="none" DEVICE="eth0" ONBOOT="yes" NM_CONTROLLED=no BRIDGE=br0 ``` ```shell $ sudo vim /etc/sysconfig/network-scripts/ifcfg-eth1 ``` ```shell DEVICE="eth1" BOOTPROTO="static" IPADDR="192.168.122.12" NETMASK="255.255.255.0" ONBOOT="yes" TYPE="Ethernet" NM_CONTROLLED="no" ``` 2. Restart the network ```shell $ sudo systemctl restart network ``` 3. Insert your host ip and host name in ==/etc/hosts== to let primary and backup node know each other. ```shell 192.168.122.11 centos2 192.168.122.12 centos3 ``` - If the hostnames are same, you need to change them first ```shell $ sudo hostnamectl set-hostname new_host_sname ``` 4. Let **primary** and **backup** node mount the NFS folder ```shell $ sudo mkdir /mnt/nfs $ sudo mount -t nfs 192.168.123.3:/home/[your nfs username]/nfsfolder /mnt/nfs ``` 5. Create a vm in **primary node**. We have created a sample xml ==cuju_vm.xml==in the cuju_libvirt_rpm folder. After modifying the part shown below in the xml file to match your image location and type, define it as a libvirt's VM. - This version does not support SPICE device and display. (We will update soon.) ```shell $ cd ~/cuju_libvirt_rpm/ $ vim cuju_vm.xml ``` ```xml= <disk type='file' device='disk'> <driver name='qemu' type='raw' cache="none"/> <source file='/mnt/nfs/Ubuntu20G-1604.img'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> ``` ```shell $ sudo virsh define cuju_vm.xml ``` ![](https://i.imgur.com/xLsUadH.png) 6. Then you can see cuju_vm is defined in libvirt ```shell sudo virsh list --cuju --all ``` ![](https://i.imgur.com/ZubQ8zj.png) After performing the above steps, you can start to test the function of Cuju. ## Cuju's Fault Tolerance ### CujuFT state explanation Here is the all stat `sudo virsh list --cuju --all` will show ![](https://i.imgur.com/LWdqgI7.png) | stat | explanation | |:----------------:|:------------------------------------------------------------------------------------------------------------------------------:| | no FT | VM has not entered FT mode after booting | | FT mode(primary) | VM is now in FT mode, located on the primary side | | FT mode(backup) | VM is now in FT mode, located on the backup side | | no backup | The VM has left the FT mode, which can be seen on the backup side after fail over or on the priamry side after executing no-FT | ### Function 1 Enter FT mode 1. Start cuju_vm vm ```shell $ sudo virsh start cuju_vm ``` ![](https://i.imgur.com/G4tQAhY.png) 2. Use VNC to see the console at the primary node ```shell $ vncviewer :5900 & ``` - **The default account/password is root/root if you use we provide guest image** 3. Check the VM status on primary and backup node ```shell $ sudo virsh list --cuju --all ``` - primary node ![](https://i.imgur.com/IUOfC9v.png) - backup node ![](https://i.imgur.com/xDh8FlA.png) 4. Start Fault Tolerance, libvirt will automatically create VM on the backup node. ```shell $ sudo virsh migrate --live --cuju cuju_vm qemu+ssh://[your backup node user name]@192.168.122.12/system ``` ![](https://i.imgur.com/JsX3EZV.png) - primary node ![](https://i.imgur.com/X3pqR07.png) - backup node ![](https://i.imgur.com/QaGmOi9.png) ### Function 2 Test fail over 1. Kill VM on primary node(Imitating vm was crashed) ```shell $ sudo kill $(ps aux | grep qemu-kvm | grep guest=cuju_vm | awk '{print $2}') ``` - Or you can use our failover instructions on backup side ```shell $ sudo virsh failover cuju_vm ``` - primary node ![](https://i.imgur.com/B5keA8x.png) - backup node ![](https://i.imgur.com/UNAeRQP.png) 2. Use VNC to see the console at the backup node ```shell $ vncviewer :5900 & ``` ### Function 3 Exit FT mode 1. If you want to exit FT mode, you can use this command, it will make primary VM exit FT mode and shut down backup VM ```shell $ sudo virsh no-ft cuju_vm ``` - primary node ![](https://i.imgur.com/3ay7jqU.png) - backup node ![](https://i.imgur.com/xDh8FlA.png)

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.