# RKE2 & Rancher 設定透過 proxy 安裝
* 在 vm 已設定 proxy server，並確認測試可以透過 proxy server 連外
## 設定 rke2
```
$ curl -sfL https://get.rke2.io --output install.sh
$ chmod +x install.sh
$ sudo mkdir -p /etc/rancher/rke2/
$ sudo vim /etc/rancher/rke2/config.yaml
node-name:
- "rms1"
token: my-shared-secret
```
* 設定 proxy
* 設定 control plane 是 `/etc/default/rke2-server` 位置
* 設定 worker 是 `/etc/default/rke2-agent` 位置
* 需要確保 Kubernetes 節點本身使用的 IP 位址範圍（即節點的公用和私有 IP）都要包含在 `NO_PROXY` 清單中
```
$ vim /etc/default/rke2-server
HTTP_PROXY=http://192.168.11.111:3128
HTTPS_PROXY=http://192.168.11.111:3128
NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,192.168.11.0/24
```
* 開始安裝 rke2
```
$ INSTALL_RKE2_CHANNEL=v1.31.8+rke2r1 ./install.sh
$ export PATH=$PATH:/opt/rke2/bin
$ systemctl enable --now rke2-server
```
* 設定 kubeconfig
```
$ mkdir .kube
$ cp /etc/rancher/rke2/rke2.yaml .kube/config
$ chown $(id -u):$(id -g) $HOME/.kube/config
$ cp /var/lib/rancher/rke2/bin/kubectl /usr/local/bin/
```
* 確認 rke2 是否成功部屬
```
$ kubectl get no
NAME STATUS ROLES AGE VERSION
rms1 Ready control-plane,etcd,master 103s v1.31.8+rke2r1
$ kubectl get po -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system cloud-controller-manager-rms1 1/1 Running 0 100s
kube-system etcd-rms1 1/1 Running 0 89s
kube-system helm-install-rke2-canal-84hsb 0/1 Completed 0 106s
kube-system helm-install-rke2-coredns-855ph 0/1 Completed 0 106s
kube-system helm-install-rke2-ingress-nginx-n6mch 0/1 Completed 0 106s
kube-system helm-install-rke2-metrics-server-5qhhp 0/1 Completed 0 106s
kube-system helm-install-rke2-runtimeclasses-gfk25 0/1 Completed 0 106s
kube-system helm-install-rke2-snapshot-controller-7csxn 0/1 Completed 1 106s
kube-system helm-install-rke2-snapshot-controller-crd-6dr88 0/1 Completed 0 106s
kube-system kube-apiserver-rms1 1/1 Running 0 101s
kube-system kube-controller-manager-rms1 1/1 Running 0 102s
kube-system kube-proxy-rms1 1/1 Running 0 71s
kube-system kube-scheduler-rms1 1/1 Running 0 102s
kube-system rke2-canal-74stb 2/2 Running 0 90s
kube-system rke2-coredns-rke2-coredns-869cb5bf57-2dzqh 1/1 Running 0 91s
kube-system rke2-coredns-rke2-coredns-autoscaler-5b89b754bd-5w8bx 1/1 Running 0 91s
kube-system rke2-ingress-nginx-controller-ln5bz 1/1 Running 0 43s
kube-system rke2-metrics-server-58ff89f9c7-2zr5g 1/1 Running 0 56s
kube-system rke2-snapshot-controller-58dbcfd956-6xf9g 1/1 Running 0 56s
```
## 安裝 Rancher
* 宣告 helm 要使用的 proxy 環境變數
```
$ export HTTP_PROXY=http://192.168.11.111:3128
$ export HTTPS_PROXY=http://192.168.11.111:3128
$ export NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,192.168.11.0/24
```
* 安裝 helm3
```
$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ ./get_helm.sh
$ helm version
version.BuildInfo{Version:"v3.17.1", GitCommit:"980d8ac1939e39138101364400756af2bdee1da5", GitTreeState:"clean", GoVersion:"go1.23.5"}
```
* 新增 rancher、cert-manager 的 helm chart
```
$ helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
$ helm repo add jetstack https://charts.jetstack.io
$ helm repo update
```
* 安裝 cert-manager
```
$ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.crds.yaml
$ helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.11.0
```
* 安裝 rancher，需注意 `noProxy` 要使用 `\` 跳脫逗號的多個位置。
* 需要確保 Kubernetes 節點本身使用的 IP 位址範圍（即節點的公用和私有 IP）包含在 `NO_PROXY` 清單中
```
$ kubectl create namespace cattle-system
$ helm install rancher rancher-stable/rancher --namespace cattle-system --set proxy=http://192.168.11.111:3128 --set noProxy="localhost\,127.0.0.1\,10.0.0.0/8\,192.168.11.0/24" --set bootstrapPassword=rancheradmin --set hostname=192.168.11.104.nip.io --set replicas=1 --version 2.10.3
```
* rancher 部屬完成
```
$ kubectl -n cattle-system get all
NAME READY STATUS RESTARTS AGE
pod/helm-operation-6fg75 0/2 Completed 0 2m5s
pod/helm-operation-b8s7n 0/2 Completed 0 2m46s
pod/helm-operation-dz6x2 0/2 Completed 0 2m18s
pod/helm-operation-jpv6v 0/2 Completed 0 92s
pod/helm-operation-n9f7l 0/2 Completed 0 57s
pod/helm-operation-r8nzf 0/2 Completed 0 45s
pod/helm-operation-skct5 0/2 Completed 0 3m8s
pod/helm-operation-skwq5 0/2 Completed 0 64s
pod/helm-operation-w46v2 0/2 Completed 0 22s
pod/rancher-5d894cd88-xg8b9 1/1 Running 0 5m52s
pod/rancher-webhook-56d5cdb4f-k77vv 1/1 Running 0 2m1s
pod/system-upgrade-controller-5fb67f585d-4hnb5 1/1 Running 0 41s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/rancher ClusterIP 10.43.54.32 <none> 80/TCP,443/TCP 5m52s
service/rancher-webhook ClusterIP 10.43.176.96 <none> 443/TCP 2m1s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/rancher 1/1 1 1 5m52s
deployment.apps/rancher-webhook 1/1 1 1 2m1s
deployment.apps/system-upgrade-controller 1/1 1 1 41s
NAME DESIRED CURRENT READY AGE
replicaset.apps/rancher-5d894cd88 1 1 1 5m52s
replicaset.apps/rancher-webhook-56d5cdb4f 1 1 1 2m1s
replicaset.apps/system-upgrade-controller-5fb67f585d 1 1 1 41s
```
## 設定 rke2 下游叢集
* 創建下游叢集時先設定 agent 環境變數，後註冊叢集
* 註冊節點後設定 control-plane 節點
```
$ vim /etc/default/rke2-server
HTTP_PROXY=http://10.10.7.54:3128
HTTPS_PROXY=http://10.10.7.54:3128
NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,10.10.7.0/24
```
```
$ cat >>/etc/systemd/system/rancher-system-agent.env<<EOF
HTTP_PROXY=http://10.10.7.54:3128
HTTPS_PROXY=http://10.10.7.54:3128
NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,10.10.7.0/24
EOF
```
```
$ sudo systemctl restart rancher-system-agent.service
```
* 設定 worker 節點
```
$ vim /etc/default/rke2-agent
HTTP_PROXY=http://10.10.7.54:3128
HTTPS_PROXY=http://10.10.7.54:3128
NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,10.10.7.0/24
```
```
$ cat >>/etc/systemd/system/rancher-system-agent.env<<EOF
HTTP_PROXY=http://10.10.7.54:3128
HTTPS_PROXY=http://10.10.7.54:3128
NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,10.10.7.0/24
EOF
```
```
$ sudo systemctl restart rancher-system-agent.service
```
## 故障排除
* 如果出現如下報錯，代表 `NO_PROXY` 有 k8s 會使用的 ip 範圍沒添加到。
```
$ kubectl -n cattle-system logs rancher-5d894cd88-zfpvj -f
Error from server: Get "https://192.168.11.104:10250/containerLogs/cattle-system/rancher-5d894cd88-zfpvj/rancher?follow=true": Forbidden
```
## 參考
https://docs.rke2.io/zh/advanced#%E9%85%8D%E7%BD%AE-http-%E4%BB%A3%E7%90%86
https://github.com/rancher/rancher/issues/16195
Sign in with Wallet
Connect another wallet