[auth] Slinky 是否一定要安裝 cert-manger ? === ###### tags: `SlinkyProject` ###### tags: `SlinkyProject`, `Kubernetes`, `k8s`, `app`, `slurm`, `SlinkyProject`, `cert-manger`, `mysql` <br> [TOC] <br> ## 討論 ### Slinky 為何需要安裝 cert-manager? 什麼情境會用到? 用途為何? - ### DeepWiki (2025/11/11) > https://deepwiki.com/search/slinky-certmanager_b4a4fbc6-88cd-40e7-8c18-59854a81fbaf 您詢問的是 Slinky 為何需要安裝 `cert-manager`,以及它的使用情境和用途。 ### 簡要說明 Slinky 需要 `cert-manager` 來為 **webhook 驗證服務**提供 TLS 憑證。 <cite>**values.yaml#132-134**</cite> Webhook 用於在 Custom Resources (CRs) 被儲存到 Kubernetes 之前進行驗證和變更。 - 參考:`helm/slurm-operator/values.yaml#132-134` ```yaml=130 # # Cert-Manager certificate configurations. certManager: # -- Enable cert-manager for certificate management. enabled: true # -- The secret to be (created and) mounted. secretName: slurm-operator-webhook-ca # -- Duration of certificate life. duration: 43800h0m0s # 5 year # -- Certificate renewal time. Should be before the expiration. ``` ### 使用情境 當您安裝 `slurm-operator` 時,系統會部署一個 webhook pod (`slurm-operator-webhook`),它實作了 Kubernetes 的 admission webhook 功能。<cite>**values.yaml#86-91**</cite> 這個 webhook 需要 TLS 憑證才能與 Kubernetes API server 進行安全通訊。 - 參考:`helm/slurm-operator/values.yaml#86-91` ```yaml=86 # Webhook configurations. webhook: # -- Enable the webhook. enabled: true # -- Set the number of replicas to deploy. replicas: 1 # -- Set the image pull policy. imagePullPolicy: IfNotPresent # -- The image to use, `${repository}:${tag}`. ``` 在安裝流程中,`cert-manager` 是第一個需要安裝的元件: <cite>**installation\.md#30-38**</cite> - 參考:`docs/installation.md#30-38` ````markdown=28 ## Slurm Operator And CRDs Install the [cert-manager] with its CRDs, if not already installed: ```sh helm repo add jetstack https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager \ --set 'crds.enabled=true' \ --namespace cert-manager --create-namespace ``` Install the slurm-operator and its CRDs: ```` ### 主要用途 #### 1. **自動憑證管理** `cert-manager` 會自動為 webhook 簽發和更新 TLS 憑證。 憑證預設有效期為 5 年,並在到期前 1 年自動更新。 - 參考:`helm/slurm-operator/values.yaml:131-139` ```yaml=130 # # Cert-Manager certificate configurations. certManager: # -- Enable cert-manager for certificate management. enabled: true # -- The secret to be (created and) mounted. secretName: slurm-operator-webhook-ca # -- Duration of certificate life. duration: 43800h0m0s # 5 year # -- Certificate renewal time. Should be before the expiration. ``` #### 2. **Webhook 驗證** Webhook 會驗證所有 Slinky CRDs 的建立和更新操作,包括:<cite></cite> - `Controller` CR (slurmctld) - `Accounting` CR (slurmdbd) - `NodeSet` CR (compute workers) - `LoginSet` CR (login nodes) - `RestApi` CR (REST API) - `Token` CR (JWT tokens) 這些驗證規則定義在 `ValidatingWebhookConfiguration` 中。 - 參考:`helm/slurm-operator/templates/webhook/webhook.yaml#29-43` ```yaml=29 apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration metadata: name: {{ include "slurm-operator.webhook.name" . }} {{- if .Values.certManager.enabled }} annotations: certmanager.k8s.io/inject-ca-from: {{ $certNamespacedName | quote }} cert-manager.io/inject-ca-from: {{ $certNamespacedName | quote }} {{- end }}{{- /* if .Values.certManager.enabled */}} labels: {{- include "slurm-operator.webhook.labels" . | nindent 4 }} webhooks: - name: accounting-v1beta1.kb.io namespaceSelector: matchExpressions: ``` #### 3. **CA 憑證注入** 當啟用 `cert-manager` 時,它會自動將 CA 憑證注入到 webhook 配置中: <cite>**webhook.yaml:33-37**</cite> - helm/slurm-operator/templates/webhook/webhook.yaml#33-37 ```yaml=29 apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration metadata: name: {{ include "slurm-operator.webhook.name" . }} {{- if .Values.certManager.enabled }} annotations: certmanager.k8s.io/inject-ca-from: {{ $certNamespacedName | quote }} cert-manager.io/inject-ca-from: {{ $certNamespacedName | quote }} {{- end }}{{- /* if .Values.certManager.enabled */}} labels: {{- include "slurm-operator.webhook.labels" . | nindent 4 }} webhooks: ``` ### 替代方案 如果您的環境中沒有 `cert-manager`,可以停用它並使用自簽憑證: - 參考:`docs/installation.md#73-83` ````markdown=73 ### Without cert-manager If the [cert-manager] is not installed, then install the chart with the `--set 'certManager.enabled=false'` argument, to avoid signing certificates via cert-manager. ```sh helm install slurm-operator oci://ghcr.io/slinkyproject/charts/slurm-operator \ --set 'certManager.enabled=false' \ --namespace=slinky --create-namespace ``` ```` 在這種情況下,Helm chart 會自動生成自簽憑證。 (<cite>**webhook.yaml#10-27**</cite>) - 參考:`helm/slurm-operator/templates/webhook/webhook.yaml#7-27` ```yaml=7 {{- $certNamespacedName := printf "%s/%s" (include "slurm-operator.namespace" .) .Values.certManager.secretName -}} {{- $durationDays := int 3650 -}} {{- $ca := genCA .Values.certManager.secretName $durationDays -}} {{- if not .Values.certManager.enabled }} {{- $cn := (include "slurm-operator.webhook.name" .) -}} {{- $dnsList := list (printf "%s.%s" (include "slurm-operator.webhook.name" .) (include "slurm-operator.namespace" .)) (printf "%s.%s.svc" (include "slurm-operator.webhook.name" .) (include "slurm-operator.namespace" .)) -}} {{- $cert := genSignedCert $cn nil $dnsList $durationDays $ca -}} --- apiVersion: v1 kind: Secret metadata: name: {{ .Values.certManager.secretName }} namespace: {{ include "slurm-operator.namespace" . }} labels: {{- include "slurm-operator.webhook.labels" . | nindent 4 }} type: kubernetes.io/tls data: tls.crt: {{ $cert.Cert | b64enc | quote }} tls.key: {{ $cert.Key | b64enc | quote }} ca.crt: {{ $ca.Cert | b64enc | quote }} {{- end }}{{- /* if not .Values.certManager.enabled */}} ``` ### Notes `cert-manager` 是 Kubernetes 生態系統中標準的憑證管理解決方案,被廣泛用於需要 TLS 憑證的服務。<cite></cite> 在 Slinky 的架構中,它專門用於保護 webhook 與 Kubernetes API server 之間的通訊,確保只有經過驗證的 Custom Resources 才能被建立或更新。<cite></cite> 這是 Kubernetes operator pattern 的標準做法。<cite></cite> Wiki pages you might want to explore: - [Overview (SlinkyProject/slurm-operator)](/wiki/SlinkyProject/slurm-operator#1) - [Architecture Overview (SlinkyProject/slurm-operator)](/wiki/SlinkyProject/slurm-operator#1.2) <br> --- ### 沒有 cert-manager 軟體,Slinky 是否能如期運作? - ### DeepWiki (2025/11/11) > https://deepwiki.com/search/slinky-certmanager_b4a4fbc6-88cd-40e7-8c18-59854a81fbaf 您詢問的是沒有 `cert-manager` 的情況下,Slinky 是否能正常運作。 ### 簡要回答 **可以運作,但需要手動配置憑證。** - `docs/installation.md#73-83` ````markdown=70 --namespace=slinky --create-namespace ``` ### Without cert-manager If the [cert-manager] is not installed, then install the chart with the `--set 'certManager.enabled=false'` argument, to avoid signing certificates via cert-manager. ```sh helm install slurm-operator oci://ghcr.io/slinkyproject/charts/slurm-operator \ --set 'certManager.enabled=false' \ --namespace=slinky --create-namespace ``` ## Slurm Cluster ```` ### 詳細說明 #### 停用 cert-manager 的安裝方式 當您停用 `cert-manager` 時,需要在安裝時設定 `certManager.enabled=false`: - `docs/installation.md#79-83` ````markdown=70 --namespace=slinky --create-namespace ``` ### Without cert-manager If the [cert-manager] is not installed, then install the chart with the `--set 'certManager.enabled=false'` argument, to avoid signing certificates via cert-manager. ```sh helm install slurm-operator oci://ghcr.io/slinkyproject/charts/slurm-operator \ --set 'certManager.enabled=false' \ --namespace=slinky --create-namespace ``` ## Slurm Cluster ```` #### 自動生成自簽憑證 當 `certManager.enabled=false` 時,Helm chart 會**自動生成自簽憑證**並建立 Secret。 <cite>**`webhook.yaml:10-26`**</cite> 這個機制使用 Helm 的 `genCA` 和 `genSignedCert` 函數來產生: <cite>**`webhook.yaml:8-13`**</cite> - `helm/slurm-operator/templates/webhook/webhook.yaml` ```yaml=6 {{- if .Values.webhook.enabled }} {{- $certNamespacedName := printf "%s/%s" (include "slurm-operator.namespace" .) .Values.certManager.secretName -}} {{- $durationDays := int 3650 -}} {{- $ca := genCA .Values.certManager.secretName $durationDays -}} {{- if not .Values.certManager.enabled }} {{- $cn := (include "slurm-operator.webhook.name" .) -}} {{- $dnsList := list (printf "%s.%s" (include "slurm-operator.webhook.name" .) (include "slurm-operator.namespace" .)) (printf "%s.%s.svc" (include "slurm-operator.webhook.name" .) (include "slurm-operator.namespace" .)) -}} {{- $cert := genSignedCert $cn nil $dnsList $durationDays $ca -}} --- apiVersion: v1 kind: Secret metadata: name: {{ .Values.certManager.secretName }} namespace: {{ include "slurm-operator.namespace" . }} labels: {{- include "slurm-operator.webhook.labels" . | nindent 4 }} type: kubernetes.io/tls data: tls.crt: {{ $cert.Cert | b64enc | quote }} tls.key: {{ $cert.Key | b64enc | quote }} ca.crt: {{ $ca.Cert | b64enc | quote }} {{- end }}{{- /* if not .Values.certManager.enabled */}} ``` - CA 憑證(有效期 10 年) - TLS 憑證和私鑰 - 自動注入 CA bundle 到 webhook 配置中 `webhook.yaml#60-62` - `helm/slurm-operator/templates/webhook/webhook.yaml` ```yaml=57 - UPDATE scope: Namespaced clientConfig: {{- if not .Values.certManager.enabled }} caBundle: {{ $ca.Cert | b64enc | quote }} {{- end }}{{- /* if not .Values.certManager.enabled */}} service: namespace: {{ include "slurm-operator.namespace" . }} name: {{ include "slurm-operator.webhook.name" . }} ``` #### Webhook 仍然需要憑證 無論是否使用 `cert-manager`,webhook pod 都**必須**掛載 TLS 憑證才能運作。 `deployment.yaml#56-72` Webhook 需要這些憑證來與 Kubernetes API server 進行安全的 HTTPS 通訊。 `main.go#107-109` #### 兩種模式的差異 | 項目 | 使用 cert-manager | 不使用 cert-manager | |------|------------------|-------------------| | **憑證來源** | cert-manager 自動簽發 | Helm 生成自簽憑證 | | **憑證更新** | 自動更新(到期前 1 年) `values.yaml#140` | 需要手動更新或重新部署 | | **CA 注入** | 自動注入 `webhook.yaml#33-36` | 手動注入 `webhook.yaml#60-61` | | **適用場景** | 生產環境 | 測試環境或無法安裝 cert-manager 的環境 | ### Notes 雖然沒有 `cert-manager` 也能運作,但**不建議在生產環境使用自簽憑證**,因為:<cite/> 1. 自簽憑證有效期固定為 10 年,無法自動更新 `webhook.yaml#8` 2. 憑證到期後需要手動重新部署 Helm chart 3. `cert-manager` 提供的自動化管理更符合 Kubernetes 最佳實踐<cite/> 開發環境中使用自簽憑證是可接受的,例如在 kind 叢集中進行本地開發時。`kind.sh#179-186` Wiki pages you might want to explore: - [Architecture Overview (SlinkyProject/slurm-operator)](/wiki/SlinkyProject/slurm-operator#1.2) - [Operator Deployment (SlinkyProject/slurm-operator)](/wiki/SlinkyProject/slurm-operator#2.1) #### Citations **File:** docs/installation.md (L73-83) ````markdown ### Without cert-manager If the [cert-manager] is not installed, then install the chart with the `--set 'certManager.enabled=false'` argument, to avoid signing certificates via cert-manager. ```sh helm install slurm-operator oci://ghcr.io/slinkyproject/charts/slurm-operator \ --set 'certManager.enabled=false' \ --namespace=slinky --create-namespace ``` ```` **File:** helm/slurm-operator/templates/webhook/webhook.yaml (L8-26) ```yaml {{- $durationDays := int 3650 -}} {{- $ca := genCA .Values.certManager.secretName $durationDays -}} {{- if not .Values.certManager.enabled }} {{- $cn := (include "slurm-operator.webhook.name" .) -}} {{- $dnsList := list (printf "%s.%s" (include "slurm-operator.webhook.name" .) (include "slurm-operator.namespace" .)) (printf "%s.%s.svc" (include "slurm-operator.webhook.name" .) (include "slurm-operator.namespace" .)) -}} {{- $cert := genSignedCert $cn nil $dnsList $durationDays $ca -}} --- apiVersion: v1 kind: Secret metadata: name: {{ .Values.certManager.secretName }} namespace: {{ include "slurm-operator.namespace" . }} labels: {{- include "slurm-operator.webhook.labels" . | nindent 4 }} type: kubernetes.io/tls data: tls.crt: {{ $cert.Cert | b64enc | quote }} tls.key: {{ $cert.Key | b64enc | quote }} ca.crt: {{ $ca.Cert | b64enc | quote }} ``` **File:** helm/slurm-operator/templates/webhook/webhook.yaml (L33-36) ```yaml {{- if .Values.certManager.enabled }} annotations: certmanager.k8s.io/inject-ca-from: {{ $certNamespacedName | quote }} cert-manager.io/inject-ca-from: {{ $certNamespacedName | quote }} ``` **File:** helm/slurm-operator/templates/webhook/webhook.yaml (L60-62) ```yaml {{- if not .Values.certManager.enabled }} caBundle: {{ $ca.Cert | b64enc | quote }} {{- end }}{{- /* if not .Values.certManager.enabled */}} ``` **File:** helm/slurm-operator/templates/webhook/deployment.yaml (L56-72) ```yaml volumeMounts: - name: certificates mountPath: /tmp/k8s-webhook-server/serving-certs/ readOnly: true {{- with .Values.webhook.affinity }} affinity: {{- toYaml . | nindent 8 }} {{- end }}{{- /* with .Values.webhook.affinity */}} {{- with .Values.webhook.tolerations }} tolerations: {{- toYaml . | nindent 8 }} {{- end }}{{- /* with .Values.webhook.tolerations */}} volumes: - name: certificates secret: defaultMode: 420 secretName: {{ .Values.certManager.secretName }} ``` **File:** cmd/webhook/main.go (L107-109) ```go WebhookServer: webhook.NewServer(webhook.Options{ TLSOpts: tlsOpts, }), ``` **File:** helm/slurm-operator/values.yaml (L140-140) ```yaml ``` **File:** hack/kind.sh (L179-186) ```shellscript function slurm-operator::prerequisites() { helm repo add jetstack https://charts.jetstack.io helm repo update helm upgrade --install cert-manager jetstack/cert-manager \ --namespace cert-manager --create-namespace \ --set 'crds.enabled=true' } ``` <br> {%hackmd vaaMgNRPS4KGJDSFG0ZE0w %}