# Rancher failed to update error messages
* 如果出現以下錯誤訊息先檢查 Upstream cluster 的`cattle-provisioning-capi-system` namespace 底下的 svc。
> failed to update fleet-default/rke2-cluster cluster.x-k8s.io/v1beta1, Kind=Cluster for rke-cluster fleet-default/rke2-cluster : Internal error occurred: failed calling webhook "default.cluster.cluster.x-k8s.io": failed to call webhook: Post "https://capi-webhook-service.cattle-provisioning-capi-system.svc:443/mutate-cluster-x-k8s-io-v1beta1-cluster?timeout=10s": no endpoints available for service "capi-webhook-service"


## 在 rms 執行
* 檢查 `capi-webhook-service` 這個 service 的 endpoint 竟然是空直

$ kubectl -n cattle-provisioning-capi-system get svc
NAME                   TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
capi-webhook-service   ClusterIP   10.43.30.98   <none>        443/TCP   72d

$ kubectl -n cattle-provisioning-capi-system get ep
NAME                   ENDPOINTS   AGE
capi-webhook-service               72d

* 檢查咬的對象是 `cluster.x-k8s.io/provider: cluster-api` 這個 label 的 pod

$ kubectl -n cattle-provisioning-capi-system get svc -o yaml | grep -A5 selector
    selector:
      cluster.x-k8s.io/provider: cluster-api
    sessionAffinity: None
    type: ClusterIP
  status:
    loadBalancer: {}

* 找到這個 label 的 pod 並且把它 redeploy

$ kubectl -n cattle-provisioning-capi-system get po -l cluster.x-k8s.io/provider=cluster-api
NAME                                       READY   STATUS    RESTARTS   AGE
capi-controller-manager-55f8866678-njc4b   1/1     Running   0          4m29s

$ kubectl -n cattle-provisioning-capi-system rollout restart deploy capi-controller-manager

* 等待 rancher 是否恢復