# 從無到有安裝BeeGFS ## 簡述 本次實驗目標在於從建立KVM Virtual Machine 開始到建立一套簡單的BeeGFS系統，配置如下 | CHAR | hostname | CPU | RAM | IP | Note | VM image | Target DISK | | ------- | ---------- | --- | --- | ----------- | --------------- | ---------------------- | ----------------------------------- | | login01 | h2-login01 | 1 | 2G | 10.7.129.52 | | chiao_h2_login01.qcow2 | | | MGMT | h2-mgmt | 2 | 4G | 10.7.129.60 | | chiao_h2_mgmt.qcow2 | | | MGS+MDS | h2-mds01 | 4 | 8G | 10.7.129.61 | 20G MDT | chiao_h2_mds01.qcow2 | chiao_h2_mds01_mdt01.qcow2 | | OSS01 | h2-ss01 | 4 | 8G | 10.7.129.62 | 5*30G OST RAID6 | chiao_h2_ss01.qcow2 | chiao_h2_ss01_ost0{1,2,3,4,5}.qcow2 | | OSS02 | hs-ss02 | 4 | 8G | 10.7.129.63 | 5*20G OST RAID6 | chiao_h2_ss02.qcow2 | chiao_h2_ss02_ost0{1,2,3,4,5}.qcow2 | | Netmask | Gateway | DNS | | ------------- | --------- | ------- | | 255.255.252.0 | 10.7.28.1 | 8.8.8.8 | ## 建立Virtual Machines 先以ISO檔安裝login01，網路設定可以選擇在安裝時以GUI設定或安裝完以後段更改網路設定部分更改 安裝時選擇最小安裝以熟悉各相容套件 ### 更改網路設定 vi /etc/sysconfig/network-scripts/ifcfg-eth0 ``` BOOTPROTO=static ONBOOT=yes IPADDR=10.7.129.## NETMASK=255.255.252.0 GATEWAY=10.7.128.1 ``` vi /etc/resolv.conf #DNS setting ``` nameserver 8.8.8.8 ``` ### 重啟網路並確認 ``` ifdown eth0 ifup eth0 ip address nmcli con show eth0 ping -c3 8.8.8.8 ``` ### 測試網路及yum ``` yum repolist yum install wget mdadm -y ``` ### 改hostname vi /etc/hostname `` %new hostname% `` ### 關閉SELINUX 以及 firewalld vi /etc/syscofig/selinux ``` SELINUX=disabled ``` ``` systemctl disable firewalld systemctl stop firewalld systemctl status firewalld ``` reboot ### 新增beegfs的repo 這邊要注意os的版本 ``` wget -O /etc/yum.repos.d/beegfs_rhel7.repo https://www.beegfs.io/release/latest-stable/dists/beegfs-rhel7.repo yum repolist ``` 到此便完成第一台VM的設置，接著便依序以複製.qcow2方式安裝其他台VM，舉例如下， ``` cd /var/lib/libvirt/images/HPC-2/ #到VM image目錄下 cp chiao_h2_login01.qcow2 chiao_h2_mgmt.qcow2 ``` 然後再以現有DISK方式建立新的VM。 新的VM開啟後記得要更改hostname以及IP ADDRESS，方法如上面敘述 ## MGMT setting ``` getenforce systemctl status firewalld yum install beegfs-mgmtd /opt/beegfs/sbin/beegfs-setup-mgmtd -p /beegfs/mgmtd systemctl start beegfs-mgmtd systemctl status beegfs-mgmtd ``` ## META setting ``` getenforce systemctl status firewalld yum install beegfs-meta mkfs.ext4 /dev/vdb mkdir -p /beegfs/mdt_01 mount /dev/vdb /beegfs/mdt_01 /opt/beegfs/sbin/beegfs-setup-meta -p /beegfs/mdt_01 -s 1 -S metadata_01_tgt_001 -m 10.7.129.60 systemctl start beegfs-meta systemctl status beegfs-meta ``` ## Storage01 setting ``` getenforce systemctl status firewalld yum install beegfs-storage lsblk #confirm disk name mdadm --create --auto=yes /dev/md0 --level=6 --raid-device=5 --spare-device=0 /dev/vd{b,c,d,e,f} mkfs -t xfs /dev/md0 mkdir -p /beegfs/OST_01 mount /dev/md0 /beegfs/OST_01 /opt/beegfs/sbin/beegfs-setup-storage -p /beegfs/OST_01 -s 100 -S storage_01_tgt_001 -i 101 -m 10.7.129.60 systemctl start beegfs-storage systemctl status beegfs-storage lsblk -f #find UUID ``` vi /etc/fstab ``` UUID=066c1049-fd5d-4da5-9c64-72666e2c2806 /beegfs/OST_01 xfs defaults 0 0 ``` ## Storage02 setting ``` getenforce systemctl status firewalld yum install beegfs-storage lsblk #confirm disk name mdadm --create --auto=yes /dev/md0 --level=6 --raid-device=5 --spare-device=0 /dev/vd{b,c,d,e,f} mkfs -t xfs /dev/md0 mkdir -p /beegfs/OST_02 mount /dev/md0 /beegfs/OST_02 /opt/beegfs/sbin/beegfs-setup-storage -p /beegfs/OST_02 -s 200 -S storage_02_tgt_001 -i 201 -m 10.7.129.60 systemctl start beegfs-storage systemctl status beegfs-storage lsblk -f #find UUID ``` vi /etc/fstab ``` UUID=066c1049-fd5d-4da5-9c64-72666e2c2806 /beegfs/OST_02 xfs defaults 0 0 ``` ## Client setting ``` getenforce systemctl status firewalld yum install beegfs-client beegfs-helperd beegfs-utils /opt/beegfs/sbin/beegfs-setup-client -m 10.7.129.60 mkdir /beegfs vi /etc/beegfs/beegfs-mounts.conf #check or change mount point to /beegfs yum insall kernel-devel-$(uname -r) #check the connection of all nodes beegfs-ctl --listnodes --nodetype=meta --nicdetails beegfs-ctl --listnodes --nodetype=storage --nicdetails systemctl start beegfs-client beegfs-helperd beegfs-net # Displays connections the client is actually using beegfs-check-servers # Displays possible connectivity of the services beegfs-df # Displays free space and inodes of storage and metadata targets ```