# Step by Step install OpenShift 4.18 with the Agent-based Installer (Online)
<style>
.indent-title-1{
margin-left: 1em;
}
.indent-title-2{
margin-left: 2em;
}
.indent-title-3{
margin-left: 3em;
}
</style>
# Preface
<div class="indent-title-1">
本篇文章會介紹，如何在多台 VM 上使用 agent-based 的方式安裝 3 台 Control-plane Node 和 3 台 Worker Node 架構的 OpenShift Container Platform 4.18
可以透過點擊展開以下目錄，選擇想看的內容，跳轉至特定章節
:::warning
:::spoiler 文章目錄
[TOC]
:::
</div>
# 1. 先備知識
## 1.1. 什麼是 Agent-based Installer
1. **Agent-based 安裝方式**是一個 OpenShift 安裝工具的子命令，可以生成一張可開機的 ISO，讓你彈性地啟動本地伺服器，自行部署叢集，甚至適用於全離線網路（air‑gapped）環境。
2. 它結合了 Assisted Installation 的便利與離線操作能力，整張 ISO 已包含部署叢集所需的所有資訊和 Release image。
3. 此安裝方式支持 Zero Touch Provisioning（ZTP），可利用宣告式方式，自動化部署裸機設備到新的邊緣站點，同時設定格式與其他安裝方式一致。
## 1.2. 理解 Agent-based Installer 安裝流程
1. 在 Control-plane node 中，其中一台會在開機一開始啟動 Assisted Service，並最終成為啟動節點（bootstrap host），這台節點被稱為 **rendezvous host（節點 0）**。
2. Assisted Service 會檢查所有節點是否符合安裝需求，並啟動 OpenShift Container Platform 叢集的部署流程；每個節點都會將 Red Hat Enterprise Linux CoreOS（RHCOS）映像寫入硬碟。
3. 除了 rendezvous host 外的其他節點會重啟並開始安裝，rendezvous host 也會在稍後重啟並加入叢集；當所有節點都完成重啟並加入時，bootstrapping 完成，叢集部署成功。
可以透過 `openshift-install agent create image` 子命令，在離線環境安裝 OpenShift Container Platform，不同架構如下：
1. **單節點 OpenShift 叢集（SNO）**
一台同時包含 master 與 worker 身份的節點。
2. **三節點緊湊型叢集**
由三台 master 節點組成，同時也是 worker 節點。
3. **高可用 OpenShift 叢集（HA）**
至少三台 master 節點，搭配任意數量的 worker 節點，具備高可用架構。
## 1.3. 各台主機扮演的角色說明
|主機名稱| 角色和服務 | 說明 |
|--------|-------------------|------------------------------------------------------------------|
|bastion| Cluster installer/bastion | 發動安裝 OCP 的跳板機, 同時提供 load balance, DNS Server 的服務 |
|master-1| rendezvous host/bootstrap | 安裝時重要的角色, 透過 installer 將 ocp cluster 角色先部署在此, 再透過 scale out 延伸到 master node |
|master-[1-3]| Master node | ocp 重要的控制節點須為三個 |
|worker-[1-3]| Worker/compute node | 執行 Application 的節點 |
|bastion| DNS Server | 提供名稱解析和反解析(網址轉成 IP 或是將 IP 轉回網址) |
|bastion| HA Proxy | 提供 load balance 的服務 |
# 2. 準備安裝
## 2.1. 先決條件
### 2.1.1. 硬體資源需求
<div class="indent-title-1">
</div>
### 2.1.2. 環境架構
請在虛擬化平台(Proxmox/VMWare...)產出以下 7 台 VM，<font color=red>並記錄每台 VM 對應的 Mac Address(除了 bastion)</font>
- 1 台 bastion (RHEL)
- 3 台 master (RHCOS)
- 3 台 worker (RHCOS)
### 2.1.3. 主機名稱設定格式
```
HOSTNAME.CLUSTER_NAME.DOMAIN_NAME
```
example:
```
bastion.topgun.kubeantony.com
```
- `HOSTNAME` 就是 `bastion`
- `CLUSTER_NAME` 就是 `topgun`
- `DOMAIN_NAME` 就是 `kubeantony.com`
### 2.1.4. 軟體授權
- Red Hat Enterprise Linux subscription
- 60 天個人免費試用連結 : https://www.redhat.com/en/technologies/linux-platforms/enterprise-linux/server/trial
- Red Hat OpenShift Container Platform subscription
- 60 天個人免費試用連結 : https://www.redhat.com/en/technologies/cloud-computing/openshift/ocp-self-managed-trial
# 3. 安裝與設定 Bastion 主機
## Step 0: 安裝與設定 Red Hat Enterprise Linux 9
- [Install Red Hat Enterprise Linux 9 - server-world](https://www.server-world.info/en/note?os=Other&p=rhel&f=5)
## Step 1: 下載 "安裝 OCP 程式" 和 "管理 OpenShift 的 CLI 工具"
### 至 [RedHat 官網連結](https://access.redhat.com/downloads/content/290/ver=4.18/rhel---9/4.18.13/x86_64/product-software) 下載以下兩個項目的檔案 (一定要有帳號)
1. **OpenShift v4.18.13 Linux Installer**
2. **OpenShift v4.18.13 Linux Client**
<div class="indent-title-1">
```!
OVERSION="4.18.13"
wget --show-progress -qO "openshift-install-linux-${OVERSION}.tar.gz" "<installer 的下載網址>" && wget --show-progress -qO "oc-${OVERSION}-linux.tar.gz" "<Clinet 的下載網址>"
```
> - `oc-4.18.13-linux.tar.gz`，是 OpenShift CLI 的壓縮檔
> - `openshift-install-linux-4.18.13.tar.gz`，是安裝 OCP 會用到的程式
> - 網址需用兩個雙引號包住，不然會報錯。
</div>
</div>
</div>
### 解壓縮檔案，並將 oc、openshift-install 和 kubectl 加至 PATH 環境變數
<div class="indent-title-1">
```!
tar -xvf oc-${OVERSION}-linux.tar.gz && \
tar -xvf openshift-install-linux-${OVERSION}.tar.gz && \
sudo mv oc kubectl openshift-install /usr/local/bin
```
螢幕輸出 :
```!
README.md
kubectl
oc
README.md
openshift-install
```
</div>
### 確認 oc Command 版本
<div class="indent-title-1">
```!
$ oc version
```
螢幕輸出 :
<pre>
Client Version: <font color=red>4.18.13</font>
Kustomize Version: <font color=blue>v5.4.2</font>
</pre>
</div>
### 確認 openshift-install Command 版本
<div class="indent-title-1">
```!
$ openshift-install version
```
螢幕輸出 :
<pre>
openshift-install <font color=red>4.18.13</font>
built from commit 9357b668a760d53a34f7094840d1e9f773127441
release image quay.io/openshift-release-dev/ocp-release@sha256:a93c65b0f9de1d2e29641fbeebc07178733db1cacc7bde178033d7b9183540bc
release architecture amd64
</pre>
</div>
### 確認 kubectl Command 版本
<div class="indent-title-1">
```!
kubectl version --client --output=yaml
```
螢幕輸出 :
<pre>
clientVersion:
buildDate: "2025-05-06T22:17:45Z"
compiler: gc
gitCommit: 35f7af703663a7459e0bc494e69ed2cc80543d04
gitTreeState: clean
gitVersion: v1.31.1
goVersion: go1.22.12 (Red Hat 1.22.12-2.el9_5) X:strictfipsruntime
major: "1"
minor: "31"
platform: linux/amd64
kustomizeVersion: <font color=blue>v5.4.2</font>
</pre>
</div>
## Step 2: 安裝 DNS Server
<div class="indent-title-1">
```
sudo yum -y install bind
```
螢幕輸出 :
```
...以上省略
Complete!
```
</div>
### 編輯 DNS Server 設定檔 named.conf
<div class="indent-title-1">
```
sudo nano /etc/named.conf
```
要改的地方有兩個部分
1. 將 `listen-on port 53` 和 `allow-query` 的值, 改成 **`any`**
<div class="indent-title-2">
檔案內容 :
<pre>
options {
listen-on port 53 { <font color=red>any</font>; };
listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
secroots-file "/var/named/data/named.secroots";
recursing-file "/var/named/data/named.recursing";
allow-query { <font color=red>any</font>; };
</pre>
</div>
2. 假設我 bastion 這台機器的 Hostname 是 `bastion.topgun.kubeantony.com`
IP 是 `192.168.11.21`，在 `/etc/named.conf` 新增以下內容至檔案的最後面 :
<div class="indent-title-2">
```
zone "kubeantony.com" {
type master;
file "/etc/named/zones/db.kubeantony.com";
};
zone "11.168.192.in-addr.arpa" {
type master;
file "/etc/named/zones/db.kubeantony.com.reverse";
};
```
:::info
在本次範例中
- Cluster name 是 `topgun`
- Base Domain Name 是 `kubeantony.com`
:::
</div>
</div>
### 設定 DNS Server 名稱解析
<div class="indent-title-1">
```!
sudo mkdir /etc/named/zones && \
sudo nano /etc/named/zones/db.kubeantony.com
```
檔案內容如下 :
<pre>
$TTL 1W
@ IN SOA <font color=red>bastion.topgun.kubeantony.com.</font> root (
2019070700 ; serial
3H ; refresh (3 hours)
30M ; retry (30 minutes)
2W ; expiry (2 weeks)
1W ) ; minimum (1 week)
IN NS <font color=red>bastion.topgun.kubeantony.com.</font>
IN MX 10 smtp.kubeantony.com.
;
;
<font color=red>bastion.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.21</font>
<font color=red>smtp.kubeantony.com.</font> IN A <font color=red>192.168.11.21</font>
;
<font color=red>api.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.21</font>
<font color=red>api-int.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.21</font>
;
<font color=red>*.apps.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.21</font>
;
<font color=red>master-1.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.23</font>
<font color=red>master-2.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.24</font>
<font color=red>master-3.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.25</font>
;
<font color=red>worker-1.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.26</font>
<font color=red>worker-2.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.27</font>
<font color=red>worker-3.topgun.kubeantony.com.</font> IN A <font color=red>192.168.11.28</font>
;
;EOF
</pre>
:::danger
**Note: 紅字部分為須依照環境的規劃來設定**
:::
</div>
### 設定 DNS Server 反解析
<div class="indent-title-1">
```!
sudo nano /etc/named/zones/db.kubeantony.com.reverse
```
檔案內容如下 :
<pre>
$TTL 1W
@ IN SOA <font color=red>bastion.topgun.kubeantony.com.</font> root (
2019070700 ; serial
3H ; refresh (3 hours)
30M ; retry (30 minutes)
2W ; expiry (2 weeks)
1W ) ; minimum (1 week)
IN NS <font color=red>bastion.topgun.kubeantony.com.</font>
;
<font color=red>21.11.168.192.in-addr.arpa.</font> IN PTR <font color=red>api.topgun.kubeantony.com.</font>
<font color=red>21.11.168.192.in-addr.arpa.</font> IN PTR <font color=red>api-int.topgun.kubeantony.com.</font>
;
<font color=red>23.11.168.192.in-addr.arpa.</font> IN PTR <font color=red>master-1.topgun.kubeantony.com.</font>
<font color=red>24.11.168.192.in-addr.arpa.</font> IN PTR <font color=red>master-2.topgun.kubeantony.com.</font>
<font color=red>25.11.168.192.in-addr.arpa.</font> IN PTR <font color=red>master-3.topgun.kubeantony.com.</font>
;
<font color=red>26.11.168.192.in-addr.arpa.</font> IN PTR <font color=red>worker-1.topgun.kubeantony.com.</font>
<font color=red>27.11.168.192.in-addr.arpa.</font> IN PTR <font color=red>worker-2.topgun.kubeantony.com.</font>
<font color=red>28.11.168.192.in-addr.arpa.</font> IN PTR <font color=red>worker-3.topgun.kubeantony.com.</font>
;
;EOF
</pre>
:::danger
**Note: 紅字部分為須依照環境的規劃來設定**
:::
</div>
### 啟動 DNS Server
<div class="indent-title-1">
```
sudo systemctl enable named --now
```
螢幕輸出 :
```!
Created symlink /etc/systemd/system/multi-user.target.wants/named.service → /usr/lib/systemd/system/named.service.
```
檢查
```
sudo systemctl status named
```
螢幕輸出 :
<pre>
● named.service - Berkeley Internet Name Domain (DNS)
Loaded: loaded (/usr/lib/systemd/system/named.service; enabled; vendor preset: disabled)
Active: <font color=grenn>active (running)</font> since Mon 2023-07-17 16:50:31 CST; 1s ago
...以下省略
</pre>
</div>
### 將 bastion 機器的 IP 新增為 DNS Server
<div class="indent-title-1">
```!
sudo nmcli connection modify 'ens18' ipv4.dns '192.168.11.21' +ipv4.dns '8.8.8.8'
sudo systemctl restart NetworkManager
```
檢查
```
sudo cat /etc/resolv.conf
```
螢幕輸出
```
# Generated by NetworkManager
search topgun.kubeantony.com
nameserver 192.168.11.21
nameserver 8.8.8.8
```
</div>
### 關閉防火牆
<div class="indent-title-1">
```
sudo systemctl disable firewalld.service --now
```
檢查是否關閉
```
sudo systemctl status firewalld.service --no-pager
```
螢幕輸出 :
```
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
...以下省略
```
</div>
### 驗證 DNS Server 正解
<div class="indent-title-1">
```
# 定義 A Zone File
AZF="/etc/named/zones/db.kubeantony.com"
# 從 DNS Zone File 中找出所有 A 記錄的主機名稱
ARecord=$(sudo grep -w 'A' $AZF | cut -d " " -f 1)
# 對每個主機名稱進行查詢，使用系統中第一個 nameserver 為查詢伺服器
for i in $ARecord; do dig @$(awk '/^nameserver/ {print $2; exit}' /etc/resolv.conf) ${i} +short; done
```
螢幕輸出 :
```
192.168.11.21
192.168.11.21
192.168.11.21
192.168.11.21
192.168.11.21
192.168.11.23
192.168.11.24
192.168.11.25
192.168.11.26
192.168.11.27
192.168.11.28
```
</div>
### 驗證 DNS Server 反解
<div class="indent-title-1">
```
# 指定 PTR Zone File 檔案路徑
PZF="/etc/named/zones/db.kubeantony.com.reverse"
# 擷取所有 PTR 記錄，將其轉換為標準 IPv4 格式，並去除重複
PTRRecord=$(sudo grep 'PTR' $PZF | awk '{split($1,a,"."); print a[4]"."a[3]"."a[2]"."a[1]}' | sort -u)
# 針對每個 IP 進行反解查詢（reverse DNS lookup），使用第一個 nameserver 查詢
for i in $PTRRecord; do dig -x ${i} @$(awk '/^nameserver/ {print $2; exit}' /etc/resolv.conf) +short; done
```
螢幕輸出 :
```
api-int.topgun.kubeantony.com.
api.topgun.kubeantony.com.
master-1.topgun.kubeantony.com.
master-2.topgun.kubeantony.com.
master-3.topgun.kubeantony.com.
worker-1.topgun.kubeantony.com.
worker-2.topgun.kubeantony.com.
worker-3.topgun.kubeantony.com.
```
</div>
## Step 3: 安裝 HAProxy 服務
<div class="indent-title-1">
```
sudo yum -y install haproxy
```
螢幕輸出 :
```
...以上省略
Complete!
```
</div>
### 設定 HAProxy
<div class="indent-title-1">
直接清掉預設的設定檔
```
cat /dev/null | sudo tee /etc/haproxy/haproxy.cfg
```
再編輯設定檔
```
sudo nano /etc/haproxy/haproxy.cfg
```
將以下內容複製到檔案中 :
<pre>
global
log 127.0.0.1 local2
pidfile /var/run/haproxy.pid
maxconn 4000
daemon
defaults
mode http
log global
option dontlognull
option http-server-close
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
listen api-server-6443
bind *:6443
mode tcp
server <font color=red>master-1 master-1.topgun.kubeantony.com:6443 check inter 1s</font>
server <font color=red>master-2 master-2.topgun.kubeantony.com:6443 check inter 1s</font>
server <font color=red>master-3 master-3.topgun.kubeantony.com:6443 check inter 1s</font>
listen machine-config-server-22623
bind *:22623
mode tcp
server <font color=red>master-1 master-1.topgun.kubeantony.com:22623 check inter 1s</font>
server <font color=red>master-2 master-2.topgun.kubeantony.com:22623 check inter 1s</font>
server <font color=red>master-3 master-3.topgun.kubeantony.com:22623 check inter 1s</font>
listen ingress-router-443
bind *:443
mode tcp
balance source
server <font color=red>worker-1 worker-1.topgun.kubeantony.com:443 check inter 1s</font>
server <font color=red>worker-2 worker-2.topgun.kubeantony.com:443 check inter 1s</font>
server <font color=red>worker-3 worker-3.topgun.kubeantony.com:443 check inter 1s</font>
listen ingress-router-80
bind *:80
mode tcp
balance source
server <font color=red>worker-1 worker-1.topgun.kubeantony.com:80 check inter 1s</font>
server <font color=red>worker-2 worker-2.topgun.kubeantony.com:80 check inter 1s</font>
server <font color=red>worker-3 worker-3.topgun.kubeantony.com:80 check inter 1s</font>
</pre>
:::danger
**Note: 紅字部分為須依照環境的規劃來設定**
:::
</div>
### 設定允許 HAProxy 可以使用 TCP Port
<div class="indent-title-1">
```
sudo setsebool -P haproxy_connect_any=1
```
> If you are using HAProxy as a load balancer and SELinux is set to **`enforcing`**, you must ensure that the HAProxy service can bind to the configured TCP port by running **`setsebool -P haproxy_connect_any=1`**.
</div>
### 啟動 HAProxy 服務，並設為開機自動啟動
<div class="indent-title-1">
```
sudo systemctl enable --now haproxy.service
```
螢幕輸出 :
```!
Created symlink /etc/systemd/system/multi-user.target.wants/haproxy.service → /usr/lib/systemd/system/haproxy.service.
```
檢查
```
sudo systemctl status haproxy.service
```
螢幕輸出 :
<pre>
● haproxy.service - HAProxy Load Balancer
Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; vendor preset: disabled)
Active: <font color=grenn>active (running)</font> since Mon 2023-07-17 17:26:21 CST; 11min ago
...以下省略
</pre>
</div>
## Step 4: 下載 pull-secret
<div class="indent-title-1">
至以下連結下載，注意 : **需先登入帳號**
- [Download Pull secret](https://console.redhat.com/openshift/create/local)
<div class="indent-title-2">
> 可點選 "**Download pull secret**"，或是點 "**Copy pull secret**" 直接將 pull-secret 複製到剪貼簿
</div>
</div>
## Step5: 編輯必要叢集設定檔
### 產生目錄
```
mkdir ~/ocp4; cd ocp4
```
### 設定 install-config.yaml
```
nano ~/ocp4/install-config.yaml
```
檔案內容如下 :
```
apiVersion: v1
baseDomain: kubeantony.com
compute:
- architecture: amd64
hyperthreading: Enabled
name: worker
replicas: 3
controlPlane:
architecture: amd64
hyperthreading: Enabled
name: master
replicas: 3
metadata:
name: topgun
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
machineNetwork:
- cidr: 192.168.11.0/24
networkType: OVNKubernetes
serviceNetwork:
- 172.30.0.0/16
platform:
none: {}
pullSecret: '{"auths":{...}'
sshKey: '...'
```
#### `install-config.yaml` 欄位介紹
| 區塊 | 鍵名 / 值 | 說明 |
| --------- | --------------------------------------- | ---------------------------------------------- |
| **版本與網域** | `apiVersion: v1` | 設定檔格式版本 |
| | `baseDomain: kubeantony.com` | 集群基礎網域，與 metadata.name 組成完整 FQDN |
| | `metadata.name: topgun` | 集群名稱，完整 FQDN 為 `topgun.kubeantony.com` |
| **控制平面** | `controlPlane.name: master` | 控制節點名稱 |
| | `controlPlane.replicas: 3` | 控制節點數量 |
| | `controlPlane.architecture: amd64` | 架構 |
| | `controlPlane.hyperthreading: Enabled` | 是否啟用超執行緒 |
| **工作節點** | `compute[0].name: worker` | 工作節點名稱 |
| | `compute[0].replicas: 3` | 工作節點數量 |
| | `compute[0].architecture: amd64` | 架構 |
| | `compute[0].hyperthreading: Enabled` | 是否啟用超執行緒 |
| **網路設定** | `networking.networkType: OVNKubernetes` | CNI 使用 OVN |
| | `clusterNetwork.cidr: 10.128.0.0/14` | Pod 網段 |
| | `clusterNetwork.hostPrefix: 23` | 每節點最大 Pod 數（約 512） |
| | `machineNetwork.cidr: 192.168.11.0/24` | 節點網路範圍 |
| | `serviceNetwork: 172.30.0.0/16` | Service 網段 |
| **平台** | `platform.none: {}` | 表示為裸機或 Agent-based 安裝 |
| **憑證與金鑰** | `pullSecret: {...}` | Red Hat container registry 認證用 |
| | `sshKey: '...'` | 安裝後 SSH 公鑰 |
### 設定 `agent-config.yaml`
```
nano agent-config.yaml
```
檔案內容如下 :
```
apiVersion: v1alpha1
kind: AgentConfig
metadata:
name: topgun
rendezvousIP: 192.168.11.13
additionalNTPSources:
- 192.168.11.11
hosts:
- hostname: master-1
role: master
interfaces:
- name: ens18
macAddress: BC:24:11:B9:6B:C1
networkConfig:
interfaces:
- name: ens18
type: ethernet
state: up
mac-address: BC:24:11:B9:6B:C1
ipv4:
enabled: true
address:
- ip: 192.168.11.13
prefix-length: 24
dhcp: false
dns-resolver:
config:
server:
- 192.168.11.11
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: 192.168.11.1
next-hop-interface: ens18
table-id: 254
rootDeviceHints:
deviceName: /dev/sda
- hostname: master-2
role: master
interfaces:
- name: ens18
macAddress: BC:24:11:5A:26:6D
networkConfig:
interfaces:
- name: ens18
type: ethernet
state: up
mac-address: BC:24:11:5A:26:6D
ipv4:
enabled: true
address:
- ip: 192.168.11.14
prefix-length: 24
dhcp: false
dns-resolver:
config:
server:
- 192.168.11.11
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: 192.168.11.1
next-hop-interface: ens18
table-id: 254
rootDeviceHints:
deviceName: /dev/sda
- hostname: master-3
role: master
interfaces:
- name: ens18
macAddress: BC:24:11:2B:C6:4B
networkConfig:
interfaces:
- name: ens18
type: ethernet
state: up
mac-address: BC:24:11:2B:C6:4B
ipv4:
enabled: true
address:
- ip: 192.168.11.15
prefix-length: 24
dhcp: false
dns-resolver:
config:
server:
- 192.168.11.11
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: 192.168.11.1
next-hop-interface: ens18
table-id: 254
rootDeviceHints:
deviceName: /dev/sda
- hostname: worker-1
role: worker
interfaces:
- name: ens18
macAddress: BC:24:11:7F:F2:B0
networkConfig:
interfaces:
- name: ens18
type: ethernet
state: up
mac-address: BC:24:11:7F:F2:B0
ipv4:
enabled: true
address:
- ip: 192.168.11.16
prefix-length: 24
dhcp: false
dns-resolver:
config:
server:
- 192.168.11.11
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: 192.168.11.1
next-hop-interface: ens18
table-id: 254
rootDeviceHints:
deviceName: /dev/sda
- hostname: worker-2
role: worker
interfaces:
- name: ens18
macAddress: BC:24:11:BE:DC:4E
networkConfig:
interfaces:
- name: ens18
type: ethernet
state: up
mac-address: BC:24:11:BE:DC:4E
ipv4:
enabled: true
address:
- ip: 192.168.11.17
prefix-length: 24
dhcp: false
dns-resolver:
config:
server:
- 192.168.11.11
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: 192.168.11.1
next-hop-interface: ens18
table-id: 254
rootDeviceHints:
deviceName: /dev/sda
- hostname: worker-3
role: worker
interfaces:
- name: ens18
macAddress: BC:24:11:05:55:16
networkConfig:
interfaces:
- name: ens18
type: ethernet
state: up
mac-address: BC:24:11:05:55:16
ipv4:
enabled: true
address:
- ip: 192.168.11.18
prefix-length: 24
dhcp: false
dns-resolver:
config:
server:
- 192.168.11.11
routes:
config:
- destination: 0.0.0.0/0
next-hop-address: 192.168.11.1
next-hop-interface: ens18
table-id: 254
rootDeviceHints:
deviceName: /dev/sda
```
### 備份設定檔
```
cp agent-config.yaml agent-config.yaml.bk
cp install-config.yaml install-config.yaml.bk
```
### 產生安裝用 ISO
需要先安裝 nmstae 套件
```
sudo yum install -y nmstate
```
再產生 ISO
```
openshift-install --dir ~/ocp4/ agent create image
```
螢幕輸出 :
```
INFO Configuration has 3 master replicas and 3 worker replicas
INFO The rendezvous host IP (node0 IP) is 192.168.1.91
INFO Extracting base ISO from release payload
INFO Verifying cached file
INFO Using cached Base ISO /home/bbg/.cache/agent/image_cache/coreos-x86_64.iso
INFO Consuming Install Config from target directory
INFO Consuming Agent Config from target directory
INFO Generated ISO at /home/bbg/ocp4/agent.x86_64.iso.
```
## Step6: 開始安裝 RedHat OpenShift
### 1. 將 ISO 上傳至虛擬化平台，並掛載到 VM 上
### 2. 在虛擬化平台將各節點透過 ISO 開機
### 3. 追蹤和驗證安裝進度 (在 Bastion 主機執行以下指令)
**確認哪些節點已安裝完畢，並需要重新開機**:
```
openshift-install --dir ~/ocp4/ agent wait-for bootstrap-complete --log-level=debug
```
:::danger
**注意! 以上指令會提示你哪台 VM 已將所有東西都安裝進硬碟裡面，然後進入自動重新開機的階段，當你看到類似的提示訊息後，大約過 30 秒 ~ 2 分鐘內，該主機會自動重開，請確保該 VM 會透過硬碟重新開機，不要再透過 ISO 又開機進去。**
:::
正確安裝的螢幕輸出如下 :
```
...
INFO Host: master-3, reached installation stage Writing image to disk: 100%
INFO Host: worker-3, reached installation stage Waiting for control plane
INFO Bootstrap Kube API Initialized
INFO Host: master-1, reached installation stage Waiting for control plane: Waiting for masters to join bootstrap control plane
INFO Uploaded logs for host master-2 cluster 84a2dfac-6f61-4de2-93e8-185d5e342f02
INFO Host: master-2, reached installation stage Rebooting
INFO Host: master-3, reached installation stage Rebooting
INFO Host: master-1, reached installation stage Waiting for bootkube
INFO Host: master-3, reached installation stage Done
INFO Node master-2 has been rebooted 1 times before completing installation
INFO Node master-3 has been rebooted 1 times before completing installation
INFO Host: worker-2, reached installation stage Rebooting
INFO Host: worker-1, reached installation stage Rebooting
INFO Host: master-1, reached installation stage Waiting for bootkube: waiting for ETCD bootstrap to be complete
INFO Bootstrap configMap status is complete
INFO Bootstrap is complete
INFO cluster bootstrap is complete
```
### 確認重開機後，OpenShift 自動安裝完成
```
openshift-install --dir ~/ocp4 agent wait-for install-complete --log-level=debug
```
螢幕輸出 :
```
...
INFO Cluster is installed
INFO Install complete!
INFO To access the cluster as the system:admin user when using 'oc', run
INFO export KUBECONFIG=/home/bbg/ocp4/auth/kubeconfig
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.topgun.kubeantony.com
INFO Login to the console with user: "kubeadmin", and password: "JyAcY-VDp6D-DYdHp-T4Twr"
```
### 設定 KubeConfig
```
mkdir ~/.kube && \
cp ~/ocp4/auth/kubeconfig ~/.kube/config
```
### 檢查叢集節點狀態
```
oc get nodes
```
螢幕輸出
```
NAME STATUS ROLES AGE VERSION
master-1 Ready control-plane,master 25m v1.31.8
master-2 Ready control-plane,master 67m v1.31.8
master-3 Ready control-plane,master 66m v1.31.8
worker-1 Ready worker 29m v1.31.8
worker-2 Ready worker 32m v1.31.8
worker-3 Ready worker 24m v1.31.8
```
### 檢視整個叢集核心元件是否健康
```
$ oc get co
```
螢幕輸出 :
```
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
authentication 4.18.13 True False False 165m
baremetal 4.18.13 True False False 3h10m
cloud-controller-manager 4.18.13 True False False 3h13m
cloud-credential 4.18.13 True False False 3h14m
cluster-autoscaler 4.18.13 True False False 3h10m
config-operator 4.18.13 True False False 3h10m
console 4.18.13 True False False 172m
control-plane-machine-set 4.18.13 True False False 3h10m
csi-snapshot-controller 4.18.13 True False False 3h2m
dns 4.18.13 True False False 76s
etcd 4.18.13 True False False 3h8m
image-registry 4.18.13 True False False 3h1m
ingress 4.18.13 True False False 59s
insights 4.18.13 True False False 3h10m
kube-apiserver 4.18.13 True False False 3h7m
kube-controller-manager 4.18.13 True False False 3h7m
kube-scheduler 4.18.13 True False False 3h6m
kube-storage-version-migrator 4.18.13 True False False 3h2m
machine-api 4.18.13 True False False 3h9m
machine-approver 4.18.13 True False False 3h10m
machine-config 4.18.13 True False False 3h10m
marketplace 4.18.13 True False False 3h10m
monitoring 4.18.13 True False False 172m
network 4.18.13 True False False 3h10m
node-tuning 4.18.13 True False False 175m
olm 4.18.13 True False False 177m
openshift-apiserver 4.18.13 True False False 3h1m
openshift-controller-manager 4.18.13 True False False 3h6m
openshift-samples 4.18.13 True False False 3h1m
operator-lifecycle-manager 4.18.13 True False False 3h9m
operator-lifecycle-manager-catalog 4.18.13 True False False 3h9m
operator-lifecycle-manager-packageserver 4.18.13 True False False 3h2m
service-ca 4.18.13 True False False 3h10m
storage 4.18.13 True False False 3h10m
```
Sign in with Wallet
Connect another wallet