OCP 4.7.6 Installation on IBM Power System === ###### tags: `IBM` `Infra` `OCP` ![](https://i.imgur.com/j8cv6rc.png) ## Bastion (192.168.30.39 & 172.16.30.39) - RHEL 7.9 ### Dnsmasq ``` yum install dnsmasq ``` /etc/dnsmasq.d/abbyland.org.conf ![](https://i.imgur.com/O5Prroc.png) /etc/dnsmasq.conf ![](https://i.imgur.com/mUxk6d7.png) /etc/resolv.conf ![](https://i.imgur.com/O7oHOIY.png) /etc/upstream_dns.conf ![](https://i.imgur.com/zOXYjXX.png) 防火牆打開 53 Port ``` firewall-cmd --add-service=dns --permanent --zone=internal firewall-cmd --reload ``` :::info firewall-cmd ```--reload``` - Current permanent configuration will become new runtime configuration. ::: 開啟 IP Forwarding ``` cat /proc/sys/net/ipv4/ip_forward 1 ``` ``` systemctl enable dnsmasq systemctl restart dnsmasq systemctl status dnsmasq ``` 驗證 DNS 功能 ``` yum install bind-utils nslookup master0.myocp.abbyland.org nslookup master1.myocp.abbyland.org nslookup master2.myocp.abbyland.org nslookup worker0.myocp.abbyland.org nslookup worker1.myocp.abbyland.org nslookup api.myocp.abbyland.org nslookup api-int.myocp.abbyland.org nslookup reg.myocp.abbyland.org ``` ### chrony ``` yum install chrony vim /etc/chrony.conf ``` ![](https://i.imgur.com/RT6OBQO.png) ``` systemctl enable chronyd systemctl restart chronyd ``` 防火牆打開 ntp 服務 ``` firewall-cmd --add-service=ntp --zone=internal --permanent firewall-cmd --reload ``` 在同網段另一台主機驗證 ``` yum install chrony vi /etc/chrony.conf ``` ![](https://i.imgur.com/c0nkVKi.png) ``` systemctl restart chronyd chronyc -a makestep chronyc sources -v ``` ### Nginx ``` # /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=https://nginx.org/packages/rhel/$releasever/$basearch/ gpgcheck=0 enabled=1 yum clean all yum repolist yum install nginx ``` /etc/nginx/conf.d/default.conf ![](https://i.imgur.com/EZXMj9U.png) ``` systemctl enable nginx systemctl restart nginx systemctl status nginx ``` 防火牆打開 8008 Port ``` firewall-cmd --add-port=8008/tcp --permanent --zone=internal firewall-cmd --reload ``` 在同網段另一台主機驗證 ``` curl 172.16.30.39:8008 ``` 往後要下載的檔案放在 /usr/share/nginx/html ### HAProxy :::info ![](https://i.imgur.com/HUwqIlU.png) ::: ``` yum install haproxy vi /etc/haproxy/haproxy.cfg ``` ![](https://i.imgur.com/p7Nglzj.png) ![](https://i.imgur.com/8H7nMx9.png) 設定 SELinux ``` setsebool -P haproxy_connect_any=1 setsebool haproxy_connect_any=1 ``` ``` systemctl enable haproxy systemctl restart haproxy systemctl status haproxy ``` 防火牆打開888, 80, 443, 6443, 22623 Port ``` firewall-cmd --add-port=888/tcp --permanent --zone=internal firewall-cmd --add-port=80/tcp --permanent --zone=internal firewall-cmd --add-port=443/tcp --permanent --zone=internal firewall-cmd --add-port=6443/tcp --permanent --zone=internal firewall-cmd --add-port=22623/tcp --permanent --zone=internal firewall-cmd --reload ``` 在同網段另一台主機驗證 ``` curl 192.168.30.39:888/status ``` ![](https://i.imgur.com/cQ2gL7o.png) ### Download Binaries 下載 [最新 openshift-install 和 oc](https://cloud.redhat.com/openshift/install/power/user-provisioned)，將```oc```和```openshift-install```命令加入PATH ``` tar xvf openshift-install-linux.tar.gz -C /usr/local/bin tar xvf openshift-client-linux.tar.gz -C /usr/local/bin ``` ```oc```版本需大於等於```openshift-install```版本 ``` openshift-install version oc version ``` 參考：[openshift-install, oc, rhcos iso, rhcos img 所有版本](https://mirror.openshift.com/pub/openshift-v4/) ### Mirror Registry #### 1. 安裝 mirror registry 產生 self-signed certificate (私鑰：/opt/registry/certs/registry.key；證書：/opt/registry/certs/registry.crt) ``` # 安裝 podman 要 enable 的 repo id subscription-manager repos --enable=rhel-7-for-power-le-extras-rpms yum install podman httpd-tools mkdir -p /opt/registry/{auth,certs,data} openssl req -newkey rsa:4096 -nodes -sha256 -keyout /opt/registry/certs/registry.key -x509 -days 365 -out /opt/registry/certs/registry.crt -subj "/C=TW/ST=Taiwan/L=Taipei/O=IBM/OU=GTS TSS/CN=reg.myocp.abbyland.org/emailAddress=abby.wu@ibm.com" ``` :::info * Country Name (2 letter code) Specify [the two-letter ISO country code](https://www.iso.org/obp/ui/#search/code/) for your location. * State or Province Name (full name) Enter the full name of your state or province. * Locality Name (eg, city) * Organization Name * Organizational Unit Name * Common Name (eg, your server’s hostname) * Email Address ::: 新增登入 mirror registry 的帳密 ``` htpasswd -bBc /opt/registry/auth/htpasswd reguser reguser1234 ``` 啟動 mirror registry 伺服器 ![](https://i.imgur.com/P5doGr8.png) ``` podman run --name mirror-registry -p 5000:5000 --restart=always \ -v /opt/registry/data:/var/lib/registry:z \ -v /opt/registry/auth:/auth:z \ -e "REGISTRY_AUTH=htpasswd" \ -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \ -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \ -v /opt/registry/certs:/certs:z \ -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/registry.crt \ -e REGISTRY_HTTP_TLS_KEY=/certs/registry.key \ -e REGISTRY_COMPATIBILITY_SCHEMA1_ENABLED=true \ -d docker.io/ibmcom/registry-ppc64le:2.7.1 podman ps -a ``` 確認 5000 Port 開啟 ``` netstat -tuanp | grep 5000 ``` 客戶端要匯入自簽憑證到受信任的根憑證授權單位 ``` cp /opt/registry/certs/registry.crt /etc/pki/ca-trust/source/anchors/ update-ca-trust trust list | head -5 ``` 在同網段另一台主機驗證 ``` curl -u reguser:reguser1234 -k https://reg.myocp.abbyland.org:5000/v2/_catalog {"repositories":[]} ``` #### 2. 製作用來下載 container image 的 pull-secret.json 下載 [pull-secret.txt](https://cloud.redhat.com/openshift/install/metal/user-provisioned)，變成易讀 pull-secret.json 檔 ``` yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm cat /etc/yum.repos.d/epel.repo yum install jq cat pull-secret.txt | jq > pull-secret.json ``` 將登入 mirror registry 所需的帳密製作成 token ``` echo -n 'reguser:reguser1234' | base64 -w0 ``` 新增 mirror registry 段落到 pull-secret.json ![](https://i.imgur.com/fduOeb6.png) #### 3. 確定 quay.io 使用的image版本，並鏡射到 mirror registry ![](https://i.imgur.com/NAFyUzp.png) 設定環境變數 ``` vi mirror_registry.env export OCP_RELEASE=4.7.6 export LOCAL_REGISTRY=reg.myocp.abbyland.org:5000 export LOCAL_REPOSITORY=ocp4/openshift4 export PRODUCT_REPO='openshift-release-dev' export SECRET_JSON=$HOME/pull-secret.json export RELEASE_NAME="ocp-release" export ARCHITECTURE=ppc64le # export REMOVABLE_MEDIA_PATH=<PATH> ``` ``` source mirror_registry.env GODEBUG=x509ignoreCN=0 oc adm -a ${SECRET_JSON} release mirror \ --from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}-${ARCHITECTURE} \ --to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \ --to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE} ``` 驗證 ``` curl -u reguser:reguser1234 -k https://reg.myocp.abbyland.org:5000/v2/_catalog {"repositories":["ocp4/openshift4"]} GODEBUG=x509ignoreCN=0 oc adm release info -a ${SECRET_JSON} "${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}" ``` :::info To use the new mirrored repository to install, add the following section to the install-config.yaml: ``` imageContentSources: - mirrors: - reg.myocp.abbyland.org:5000/ocp4/openshift4 source: quay.io/openshift-release-dev/ocp-release - mirrors: - reg.myocp.abbyland.org:5000/ocp4/openshift4 source: quay.io/openshift-release-dev/ocp-v4.0-art-dev ``` To use the new mirrored repository for upgrades, use the following to create an ImageContentSourcePolicy: ``` apiVersion: operator.openshift.io/v1alpha1 kind: ImageContentSourcePolicy metadata: name: example spec: repositoryDigestMirrors: - mirrors: - reg.myocp.abbyland.org:5000/ocp4/openshift4 source: quay.io/openshift-release-dev/ocp-release - mirrors: - reg.myocp.abbyland.org:5000/ocp4/openshift4 source: quay.io/openshift-release-dev/ocp-v4.0-art-dev ``` ::: ### NFS Server (可選) 架設 NFS Server 用來給安裝完成後 image-registry Pod 配置 PV ``` yum install nfs-utils mkdir -p /var/ocp/image-registry chmod -R 777 /var/ocp/image-registry echo "/var/ocp/image-registry 172.16.30.0/24(rw,sync,no_root_squash,no_all_squash)" >> /etc/exports systemctl enable nfs-server systemctl restart nfs-server systemctl enable rpcbind systemctl restart rpcbind ``` 開啟防火牆與SELinux ``` firewall-cmd --permanent --zone internal --add-service rpc-bind firewall-cmd --permanent --zone internal --add-service mountd firewall-cmd --permanent --zone internal --add-service nfs firewall-cmd --reload setsebool -P nfs_export_all_rw on setsebool nfs_export_all_rw on setsebool -P nfs_export_all_ro on setsebool nfs_export_all_ro on ``` 在172.16.30.0/24網段，Client端測試 ``` showmount -e 172.16.30.39 ``` ### Firewall 確認防火牆設定 ``` firewall-cmd --list-all-zones ``` ![](https://i.imgur.com/ZGltTMl.png) ### install-config.yml * pull-secret.json - 去掉空白處和隔行 ``` cat pull-secret.json | awk '{print}' ORS='' | sed -E 's/ //g' > pull-secret.json.squeeze ``` * .ssh/id_rsa - 保留好成對的公私鑰 .ssh/id_rsa，往後要貼 id_rsa.pub 到 install-config.yml，將來能以 core 帳號登入其他節點 ``` ssh-keygen -t rsa -b 4069 -f /root/.ssh/id_rsa ``` ``` mkdir /root/ocp-installer vi /root/ocp-installer/install-config.yml ``` ![](https://i.imgur.com/tqGqbtD.png) ![](https://i.imgur.com/rWKDLOE.png) ::: danger 注意: 後續產生 Kubernetes manifest，install-config.yaml 會被消耗掉，如需安裝多次，務必備份！ ``` cp /root/ocp-installer/install-config.yaml /root/back_up/install-config.yml ``` ::: ### 產生 Kubernetes manifest ``` openshift-install create manifests --dir=/root/ocp-installer ``` 客製化清單 * 讓 master node 不要被安排 pod ``` sed -i 's/mastersSchedulable: true/\mastersSchedulable: false/' /root/ocp-installer/manifests/cluster-scheduler-02-config.yml ``` * 對 master 和 worker node 設定 ntp client ``` cat << EOF | base64 server ntp.myocp.abbyland.org iburst driftfile /var/lib/chrony/drift makestep 1.0 3 rtcsync logdir /var/log/chrony EOF ``` ``` ICAgIHNlcnZlciBudHAubXlvY3AuYWJieWxhbmQub3JnIGlidXJzdCAKICAgIGRyaWZ0ZmlsZSAvdmFyL2xpYi9jaHJvbnkvZHJpZnQKICAgIG1ha2VzdGVwIDEuMCAzCiAgICBydGNzeW5jCiAgICBsb2dkaXIgL3Zhci9sb2cvY2hyb255Cg== ``` 編輯 /root/ocp-installer/openshift/99-workers-chrony-configuration.yaml 與 /root/ocp-installer/openshift/99-masters-chrony-configuration.yaml，貼上 base64 編碼 ``` cat << EOF > /root/ocp-installer/openshift/99-workers-chrony-configuration.yaml apiVersion: machineconfiguration.openshift.io/v1 kind: MachineConfig metadata: labels: machineconfiguration.openshift.io/role: worker # 另一份為 master name: 99-workers-chrony-configuration # 另一份為 99-masters-chrony-configuration spec: config: ignition: config: {} security: tls: {} timeouts: {} version: 2.2.0 networkd: {} passwd: {} storage: files: - contents: source: data:text/plain;charset=utf-8;base64,ICAgIHNlcnZlciBudHAubXlvY3AuYWJieWxhbmQub3JnIGlidXJzdCAKICAgIGRyaWZ0ZmlsZSAvdmFyL2xpYi9jaHJvbnkvZHJpZnQKICAgIG1ha2VzdGVwIDEuMCAzCiAgICBydGNzeW5jCiAgICBsb2dkaXIgL3Zhci9sb2cvY2hyb255Cg== verification: {} filesystem: root mode: 420 path: /etc/chrony.conf osImageURL: "" EOF ``` 最後得到清單為 ``` yum install tree tree -a ocp-installer/ ``` ![](https://i.imgur.com/L9T6xAe.png) :::danger 注意: 備份客製化清單 ``` cp /root/ocp-installer/openshift/99-masters-chrony-configuration.yaml /root/back_up/99-masters-chrony-configuration.yaml cp /root/ocp-installer/openshift/99-workers-chrony-configuration.yaml /root/back_up/99-workers-chrony-configuration.yaml ``` ::: ### 產生 ignition 檔案 :::danger 注意: The Ignition config files that the installation program generates contain certificates that expire after 24 hours. You must complete your cluster installation and keep the cluster running for 24 hours in a non-degraded state to ensure that the first certificate rotation has finished. ::: ``` openshift-install create ignition-configs --dir=/root/ocp-installer tree -a ocp-installer/ ``` ![](https://i.imgur.com/4EdlE4Z.png) ``` cp /root/ocp-installer/*.ign /usr/share/nginx/html/ chmod a+r /usr/share/nginx/html/*.ign ``` 在同網段，另一台主機測試 ``` curl http://172.16.30.39:8008/worker.ign ``` ::: danger 注意: 重新安裝時，務必刪除如下檔案，及隱藏檔.openshift_install.log、.openshift_install_state.json ``` cd /root/ocp-installer/ rm -fr *.ign meta*.json auth rm -f .openshift_install.log rm -f .openshift_install_state.json ``` ::: ### 放置 image URL 下載 [rhcos-4.7.0-ppc64le-live-rootfs.ppc64le.img](https://mirror.openshift.com/pub/openshift-v4/ppc64le/dependencies/rhcos/4.7/4.7.0/) ，放到 nginx ``` cp rhcos-4.7.0-ppc64le-live-rootfs.ppc64le.img /usr/share/nginx/html/ chmod a+r rhcos-4.7.0-ppc64le-live-rootfs.ppc64le.img ``` 在同網段，另一台主機測試 ``` curl http://172.16.30.39:8008/rhcos-4.7.0-ppc64le-live-rootfs.ppc64le.img ``` --- ## Bootstrap (172.16.30.40), Master (172.16.30.41-43), Worker (172.16.30.44-45) - RHCOS 4.7.0 將 [RHCOS ISO](https://cloud.redhat.com/openshift/install/power/user-provisioned) 掛到開機硬碟。 :::danger 注意: The RHCOS images might not change with every release of OpenShift Container Platform. You must download images with the highest version that is less than or equal to the OpenShift Container Platform version that you install. ::: ### 驗證網路功能 (除錯) 以下僅示範 bootstrap，其餘 node 均須執行相同步驟 ``` sudo nmtui ``` #### 主機名稱 ![](https://i.imgur.com/Pu3CIQr.png) #### vNIC1 - 用來上網或除錯 (可選) Address (192.168.30.40/24) Gateway (192.168.30.254) ![](https://i.imgur.com/kQRtmIh.png) ![](https://i.imgur.com/YduT27f.png) 確認網路環境 ``` ping -I env4 8.8.8.8 ``` #### vNIC2 - 用來連通 cluster 其他節點 (必要) Address (172.16.30.40/24) Gateway (不要設) DNS (172.16.30.39) ![](https://i.imgur.com/ysrAc5m.png) ![](https://i.imgur.com/WKlFqCs.png) 確認網路環境 * 確認 curl 得到 nginx 服務 * 確認 DNS * 確認 curl 得到 mirror-registry 服務 ``` [core@localhost ~]$ curl 172.16.30.39:8008 . . <h1>Welcome to nginx!</h1> . . [core@localhost ~]$ nslookup master0 Server: 172.16.30.39 Address:172.16.30.39#53 Name: master0.myocp.abbyland.org Address: 172.16.30.41 [core@localhost ~]$ curl -u reguser:reguser1234 -k https://reg.myocp.abbyland.org:5000/v2/_catalog {"repositories":["ocp4/openshift4"]} ``` ### 驗證 NTP Client 功能 (除錯) ``` sudo timedatectl set-timezone "Asia/Taipei" sudo vi /etc/chrony.conf ``` ![](https://i.imgur.com/c0nkVKi.png) ``` sudo systemctl restart chronyd sudo chronyc -a makestep sudo chronyc sources -v ``` --- ## Install Control Plane Nodes (Bootstrap & Master Nodes) 依序是先對 Bootstrap 執行安裝動作，再對 Master nodes 執行安裝動作，等待 Bootstrap 完成移交控制權給 Master nodes，再對 Worker nodes 執行安裝動作 * Bootstrap 在 RHCOS live 的 GRUB 選單按 ```e``` 去編輯 kernel 參數 ``` coreos.inst=yes coreos.inst.install_dev=sda dfltcc=off coreos.inst.insecure coreos.live.rootfs_url=http://172.16.30.39:8008/rhcos-live-rootfs.ppc64le.img coreos.inst.ignition_url=http://172.16.30.39:8008/bootstrap.ign ip=172.16.30.40::172.16.30.39:255.255.255.0:bootstrap.myocp.abbyland.org:env5:none nameserver=172.16.30.39 ``` :::info Node’s IP address to 172.16.30.40 Gateway address to 172.16.30.39 Netmask to 255.255.255.0 Hostname to bootstrap.myocp.abbyland.org The DNS server address to 172.16.30.39 ::: 把外網的nameserver註解掉 ``` [core@bastion ~]$ sudo vi /etc/resolv.conf [core@bastion ~]$ sudo vi /etc/NetworkManager/NetworkManager.conf [main] dns=none [core@bastion ~]$ sudo systemctl restart NetworkManager [core@bastion ~]$ less /etc/resolv.conf ``` * Master Nodes ``` coreos.inst=yes coreos.inst.install_dev=sda dfltcc=off coreos.inst.insecure coreos.live.rootfs_url=http://172.16.30.39:8008/rhcos-live-rootfs.ppc64le.img coreos.inst.ignition_url=http://172.16.30.39:8008/master.ign ip=172.16.30.41::172.16.30.39:255.255.255.0:master0.myocp.abbyland.org:env5:none nameserver=172.16.30.39 ``` ``` coreos.inst=yes coreos.inst.install_dev=sda dfltcc=off coreos.inst.insecure coreos.live.rootfs_url=http://172.16.30.39:8008/rhcos-live-rootfs.ppc64le.img coreos.inst.ignition_url=http://172.16.30.39:8008/master.ign ip=172.16.30.42::172.16.30.39:255.255.255.0:master1.myocp.abbyland.org:env5:none nameserver=172.16.30.39 ``` ``` coreos.inst=yes coreos.inst.install_dev=sda dfltcc=off coreos.inst.insecure coreos.live.rootfs_url=http://172.16.30.39:8008/rhcos-live-rootfs.ppc64le.img coreos.inst.ignition_url=http://172.16.30.39:8008/master.ign ip=172.16.30.43::172.16.30.39:255.255.255.0:master2.myocp.abbyland.org:env5:none nameserver=172.16.30.39 ``` 把外網的nameserver註解掉 ``` [core@master# ~]$ sudo vi /etc/resolv.conf [core@master# ~]$ sudo vi /etc/NetworkManager/NetworkManager.conf [main] dns=none [core@master# ~]$ sudo systemctl restart NetworkManager [core@master# ~]$ less /etc/resolv.conf ``` * Bastion 監視 Bootstrap 是否完成移交控制權給 Master nodes ``` ssh core@bootstrap journalctl -b -f -u release-image.service -u bootkube.service ``` 再開一個 Bastion terminal ``` openshift-install --dir=/root/ocp-installer wait-for bootstrap-complete --log-level=debug ``` Bootstrap 關機 --- ## Install Worker Nodes * Worker Nodes ``` coreos.inst=yes coreos.inst.install_dev=sda dfltcc=off coreos.inst.insecure coreos.live.rootfs_url=http://172.16.30.39:8008/rhcos-live-rootfs.ppc64le.img coreos.inst.ignition_url=http://172.16.30.39:8008/worker.ign ip=172.16.30.44::172.16.30.39:255.255.255.0:worker0.myocp.abbyland.org:env5:none nameserver=172.16.30.39 ``` ``` coreos.inst=yes coreos.inst.install_dev=sda dfltcc=off coreos.inst.insecure coreos.live.rootfs_url=http://172.16.30.39:8008/rhcos-live-rootfs.ppc64le.img coreos.inst.ignition_url=http://172.16.30.39:8008/worker.ign ip=172.16.30.45::172.16.30.39:255.255.255.0:worker1.myocp.abbyland.org:env5:none nameserver=172.16.30.39 ``` 把外網的nameserver註解掉 ``` [core@worker# ~]$ sudo vi /etc/resolv.conf [core@worker# ~]$ sudo vi /etc/NetworkManager/NetworkManager.conf [main] dns=none [core@worker# ~]$ sudo systemctl restart NetworkManager [core@worker# ~]$ less /etc/resolv.conf ``` * Bastion ``` export KUBECONFIG=/root/ocp-installer/auth/kubeconfig watch oc get csr -A oc get csr -A -o name |xargs oc adm certificate approve ``` ``` watch oc get clusteroperators ``` :::info oc -A, --all-namespaces -o, --output='': Output format. One of: json|yaml|wide|name|custom-columns=...|custom-columns-file=...|go-template=...|go-template-file=...|jsonpath=...|jsonpath-file=... ::: 監視 OCP 是否完成安裝 ``` openshift-install --dir=/root/ocp-installer wait-for install-complete --log-level=debug ``` --- ## References - [Initramfs 機制](http://blog.linux.org.tw/~jserv/archives/001954.html) - [ How to create a local YUM repository](https://access.redhat.com/solutions/23016) - [What is a DNS SRV record?](https://www.cloudflare.com/en-in/learning/dns/dns-records/dns-srv-record/) - [Use dnsmasq to provide DNS & DHCP services](https://fedoramagazine.org/dnsmasq-provide-dns-dhcp-services/) - [Install mirror registry](https://blog.csdn.net/weixin_43902588/article/details/107727804) - [Configure external image registry, internal registry, mirror registry](https://blog.csdn.net/weixin_43902588/article/details/107710166) - [Network Definitions of install-config.yaml for OpenShift 4 on IBM Z and IBM LinuxONE](https://medium.com/@filipe.miranda/understanding-network-definitions-from-install-config-yaml-b3673a7c220f) - [在 CentOS7/RHEL7 設定 NFS服務](https://linux.incomeself.com/%E5%9C%A8-centos7-rhel7-%E8%A8%AD%E5%AE%9A-nfs%E6%9C%8D%E5%8B%99/) - [Installing a cluster on IBM Power Systems](https://docs.openshift.com/container-platform/4.7/installing/installing_ibm_power/installing-ibm-power.html) - [Red Hat OpenShift Container Platform 安装体验 基于 OpenShift 容器云平台的解决方案](https://csc.cn.ibm.com/src/index/425ebe3f-163d-4bd2-8ffd-a3770f9f5162?rtype=experiment&roadmapId=5712290d-705d-4c76-993d-72088e851619&eventId=0c6637e0-e9c6-461c-a93d-b362f4cd1fd8) - [Customizing nodes - Configuring chrony time service](https://docs.openshift.com/container-platform/4.5/installing/install_config/installing-customizing.html#installation-special-config-chrony_installing-customizing) - HAProxy ![](https://i.imgur.com/1yZTGq0.png)