**如何對您的 Kubernetes 設置使用者權限?** **快速帶您部署OPA在您GKE的K8s叢集!** === ![](https://i.imgur.com/4ubGLXl.jpg) <style> .black { color: black; } </style> ###### tags: `GCP` ## **OPA是什麼?** :::info <span class="black">在我們的生活中許多事物都會運用到策略，例如:近年因為covid-19大流行，各國政府就都制定了個別的防疫策略。當我們把策略套用到組織上時，組織通常需要在其運行應用程序的環境中應用各種策略。這些策略可能需要滿足符合規定的要求、實現更高的安全性並且可以跨多個環境的策略標準。這需要一種方式來定義和執行這些策略。這時候像 OPA 這樣的策略代理就可以幫助我們實現同樣的策略管理目標。 </span> ::: ## **為什麼要使用OPA?** :::info <span class="black">當我們運行我們的應用程序時，它通常包含多個服務同時運行。即使在最簡單的架構下，我們也將擁有一個 API gateway/load balancer, 1-2個應用程式，1個資料庫。通常，所有這些服務都會有不同的請求授權機制，例如，應用程序可能正在使用 JWT tokens來授權請求，您的資料庫也正在使用授權來授權請求，您的應用程序也可能正在訪問一些第三方 API 或雲服務將再次以不同的方式授權請求。或是將您的 CI/CD 服務器、日誌服務器等添加到您的k8s叢集內，您可以發現即使在小型架構中也存在多種不同的授權方式。 當我們需要滿足合規性或信息安全要求，甚至是一些自訂義比較嚴謹的組織策略時，我們系統中存在如此多的授權模型使得統一授權變得困難。例如，如果我們需要遵守一些新的合規性要求，那麼我們需要對系統中需要進行授權的所有組件都了解並套用相同的策略。</span> ::: ## **OPA如何執行控管 Kubernetes 策略?** :::info <span class="black">在 Kubernetes 中，Admission control在創建、更新和刪除操作期間對對象實施策略。每次向集群發送 API 請求以創建/更新/刪除資源時，Gatekeeper 使用admission controller webhook 來執行策略。</span> ::: ### **Gatekeeper 基於 OPA 約束框架：** ●約束模板(Rego 代碼) ●約束 ![](https://i.imgur.com/1TeJjuv.jpg) :::info <span class="black">當您在使用Kubernetes進行任何操作時，都會先發送Request到Kubernetes的server在傳遞至Kubernetes API，API Server 將 webhook 請求中的整個 Kubernetes 對象發送給 OPA，任何需要做出授權或策略決策的服務都可以調用（策略查詢），然後 OPA 可以根據策略的 Rego 代碼做出決策，並將決策返回給相應地處理請求的服務. 執行則由實際服務本身完成，OPA 僅負責做出決定。</span> ::: 相關說明: [webhook](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks "游標顯示") [Admission Controllers](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/ "游標顯示") --- ## **在 Kubernetes 中進行演示** 因採用GCP進行演示，開始實作之前需確認您的環境符合以下條件 <style> .red { color: red; } </style> :::success <span class="red">**1.您必須先有一個kubernetes叢集在Google Cloud上**</span> <span class="red">**2.在kubernetes叢集中創建資源的管理權限**</span> ::: --- <style> .blue { color: blue; } </style> ### <span class="blue">**實作限制deployment的replicas數量**</span> #### <span class="red">**※需要注意GKE版本是否正確，本篇使用1.21.11進行演示**</span> 首先需要透過GCP的 Cloud Shell 登入您的GKE叢集 ![](https://i.imgur.com/6dFGfSV.png) 進入您的 Cloud Shell 後複製指令執行，就可以進入您GCP上的GKE ``` gcloud container clusters get-credentials cluster-1 --zone us-central1-c --project <your-project-ID> ``` 顯示如下 ![](https://i.imgur.com/F4kj8WV.png) 開始設定OPA之前您需要先部署Gatekeeper在k8s上，需先執行以下指令 ``` kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/release-3.6/deploy/gatekeeper.yaml ``` 執行完成後，您可以看到輸出如下都成功created代表Gatekeeper已經建立在您的GKE叢集 ``` namespace/gatekeeper-system created resourcequota/gatekeeper-critical-pods created customresourcedefinition.apiextensions.k8s.io/configs.config.gatekeeper.sh created customresourcedefinition.apiextensions.k8s.io/constraintpodstatuses.status.gatekeeper.sh created customresourcedefinition.apiextensions.k8s.io/constrainttemplatepodstatuses.status.gatekeeper.sh created customresourcedefinition.apiextensions.k8s.io/constrainttemplates.templates.gatekeeper.sh created serviceaccount/gatekeeper-admin created Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+ podsecuritypolicy.policy/gatekeeper-admin created role.rbac.authorization.k8s.io/gatekeeper-manager-role created clusterrole.rbac.authorization.k8s.io/gatekeeper-manager-role created rolebinding.rbac.authorization.k8s.io/gatekeeper-manager-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/gatekeeper-manager-rolebinding created secret/gatekeeper-webhook-server-cert created service/gatekeeper-webhook-service created deployment.apps/gatekeeper-audit created deployment.apps/gatekeeper-controller-manager created Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget poddisruptionbudget.policy/gatekeeper-controller-manager created validatingwebhookconfiguration.admissionregistration.k8s.io/gatekeeper-validating-webhook-configuration created ``` <style> .green { color: green; } </style> <span class="red">※Warning的提示是說明GKE新的版本要執行其他程式碼需要透過此[文件說明](https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke)操作進行授權</span> 現在先編寫一個Constraint template的yaml檔 ``` apiVersion: templates.gatekeeper.sh/v1 kind: ConstraintTemplate metadata: name: k8sreplicalimits annotations: description: >- Requires that objects with the field `spec.replicas` (Deployments, ReplicaSets, etc.) specify a number of replicas within defined ranges. spec: crd: spec: names: kind: K8sReplicaLimits validation: # Schema for the `parameters` field openAPIV3Schema: type: object properties: ranges: type: array description: Allowed ranges for numbers of replicas. Values are inclusive. items: type: object description: A range of allowed replicas. Values are inclusive. properties: min_replicas: description: The minimum number of replicas allowed, inclusive. type: integer max_replicas: description: The maximum number of replicas allowed, inclusive. type: integer targets: - target: admission.k8s.gatekeeper.sh rego: | package k8sreplicalimits deployment_name = input.review.object.metadata.name violation[{"msg": msg}] { spec := input.review.object.spec not input_replica_limit(spec) msg := sprintf("The provided number of replicas is not allowed for deployment: %v. Allowed ranges: %v", [deployment_name, input.parameters]) } input_replica_limit(spec) { provided := input.review.object.spec.replicas count(input.parameters.ranges) > 0 range := input.parameters.ranges[_] value_within_range(range, provided) } value_within_range(range, value) { range.min_replicas <= value range.max_replicas >= value } ``` 編寫完yaml檔後，現在建立約束的模板 ``` kubectl apply -f template.yaml 建立後輸出如下，代表約束的模板已建立完成 constrainttemplate.templates.gatekeeper.sh/k8sreplicalimits created ``` 約束的模板建立完成後，編寫要執行約束的constraint.yaml ``` apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sReplicaLimits metadata: name: replica-limits spec: match: kinds: - apiGroups: ["apps"] kinds: ["Deployment"] parameters: ranges: - min_replicas: 3 max_replicas: 50 ``` 編寫完yaml後，現在建立約束 ``` kubectl apply -f constraint.yaml 建立後輸出如下，代表約束已建立完成 k8sreplicalimits.constraints.gatekeeper.sh/replica-limits created ``` 接著建立測試約束是否生效在Deployment的yaml <span class="blue">example_allowed.yaml</span> ``` apiVersion: apps/v1 kind: Deployment metadata: name: allowed-deployment spec: selector: matchLabels: app: nginx replicas: 3 template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.14.2 ports: - containerPort: 80 ``` <span class="blue">example_disallowed.yaml</span> ``` apiVersion: apps/v1 kind: Deployment metadata: name: disallowed-deployment spec: selector: matchLabels: app: nginx replicas: 100 template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.14.2 ports: - containerPort: 80 ``` 編寫完yaml後，直接建立測試Deployment ``` 執行example_allowed.yaml後，因為replicas在約束的範圍內，因此可以正常建立，輸出內容如下 alvin_yang@cloudshell:~ (your-project-ID)$ kubectl apply -f example_allowed.yaml deployment.apps/allowed-deployment created alvin_yang@cloudshell:~ (your-project-ID)$ kubectl get deployment NAME READY UP-TO-DATE AVAILABLE AGE allowed-deployment 3/3 3 3 17s ``` ``` 執行example_disallowed.yaml後，因為replicas在約束的範圍外，會受到限制無法建立 alvin_yang@cloudshell:~ (your-project-ID)$ kubectl apply -f example_disallowed.yaml ``` :::danger 會輸出以下錯誤訊息 Error from server ([replica-limits] The provided number of replicas is not allowed for deployment: disallowed-deployment.Allowed ranges: {"ranges": [{"max_replicas": 50, "min_replicas": 3}]}):error when creating "example_disallowed.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [replica-limits] The provided number of replicas is not allowed for deployment: disallowed-deployment. Allowed ranges: {"ranges": [{"max_replicas": 50, "min_replicas": 3}]} ::: :::warning 從上面的錯誤訊息可以得知先由Kubernetes報錯無法建立這個Deployment，後面的錯誤訊息則是Gatekeeper拒絕這個建立Deployment的Request，因為測試的Deployment透過Replicas建立的Pod數量超過了Rego Code約束的範圍，所以請求會遭到拒絕。 ::: 相關說明 [Gatekeeper](https://kubernetes.io/blog/2019/08/06/opa-gatekeeper-policy-and-governance-for-kubernetes/) ## 總結 在上面的簡單示範中，我們透過了GKE來展示如何在GKE透過Open Policy Agent制定約束的模板來執行政策的控管和套用，關於Open Policy Agent的更多資訊可以參考[Open Policy Agent](https://www.openpolicyagent.org/docs/latest/kubernetes-introduction/)的官方網站來進行更深入的了解和應用，讓您能透過Open Policy Agent來輕鬆的管理您的Kubernetes叢集。