# HashiCorp Vault Secret Protect ## Vault 使用背景 Kubernetes 原生 Secret 機制預設僅採用 Base64 編碼,這僅是一種數據格式轉換而非加密技術。這意味著任何擁有 Namespace 讀取權限的人員或程序,都能輕易將其還原為明文,缺乏足夠強度的資安防護機制。 而透過 Vault 的 Agent Sidecar Injector 功能,可以將 secret 內容儲存在 Vault 內,透過 Vault 的保護機制來確保 secret 不被竊取。 ## 安裝 Vault ``` $ helm repo add hashicorp https://helm.releases.hashicorp.com $ helm search repo hashicorp/vault NAME CHART VERSION APP VERSION DESCRIPTION hashicorp/vault 0.31.0 1.20.4 Official HashiCorp Vault Chart hashicorp/vault-secrets-gateway 0.0.2 0.1.0 A Helm chart for Kubernetes hashicorp/vault-secrets-operator 1.0.1 1.0.1 Official Vault Secrets Operator Chart ``` * K8s 環境已準備好 `nfs-csi` csi,或是使用自己環境的也可以。 ``` $ kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE nfs-csi nfs.csi.k8s.io Retain Immediate false 35d ``` * 安裝 vault 設定資料永存,這裡需要指定自己環境的 csi ``` $ helm install vault hashicorp/vault \ --create-namespace \ --set='ui.enabled=true' \ --set='ui.serviceType=NodePort' \ --set='server.dataStorage.storageClass=nfs-csi' \ --set='server.dataStorage.accessMode=ReadWriteMany' \ --set='server.dataStorage.size=20Gi' \ --namespace vault ``` * 這時 vault pod 的 container 是還沒有準備好的,我們還需要手動初始化他 ``` $ kubectl -n vault get all NAME READY STATUS RESTARTS AGE pod/vault-0 0/1 Running 0 10s pod/vault-agent-injector-556c5dd8fb-8g4bd 1/1 Running 0 11s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/vault ClusterIP 10.96.33.246 <none> 8200/TCP,8201/TCP 11s service/vault-agent-injector-svc ClusterIP 10.96.243.132 <none> 443/TCP 11s service/vault-internal ClusterIP None <none> 8200/TCP,8201/TCP 11s service/vault-ui NodePort 10.96.1.24 <none> 8200:30768/TCP 11s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/vault-agent-injector 1/1 1 1 11s NAME DESIRED CURRENT READY AGE replicaset.apps/vault-agent-injector-556c5dd8fb 1 1 1 11s NAME READY AGE statefulset.apps/vault 0/1 11s ``` * 初始化 vault,這邊的 Unseal Key 需要自己保留,如果 vault pod 重啟,需要透過以下的 key 來解鎖。 ``` $ kubectl exec -ti vault-0 -n vault -- /bin/sh / $ vault operator init Unseal Key 1: TxGdasK3Wt5ptBw/YHOmE4ZfnyICVusP2eK0NBk9MDRX Unseal Key 2: +VVeL1qtFH1L3euMTocIPrD28PbhtlLIrpyDO25w1aW+ Unseal Key 3: 9YEh9Lq0Pgc5TU4EMGTjV2o+88vgC/azrc+vm5GOz3UY Unseal Key 4: C2MZTuwxxHaa3cAcFi7BIAd41D/JQz4BSVDu42k+kz3y Unseal Key 5: zYUxWqhDz96nU5iO7pYkS1jRndfU3fBq319v0QKv/sNL Initial Root Token: hvs.lmGfDNMDgpl6uxb39exPTg4N Vault initialized with 5 key shares and a key threshold of 3. Please securely distribute the key shares printed above. When the Vault is re-sealed, restarted, or stopped, you must supply at least 3 of these keys to unseal it before it can start servicing requests. Vault does not store the generated root key. Without at least 3 keys to reconstruct the root key, Vault will remain permanently sealed! It is possible to generate new unseal keys, provided you have a quorum of existing unseal keys shares. See "vault operator rekey" for more information. # 輸入其中的三把 key 用來解鎖 Shamir's Secret Sharing 的演算法 / $ vault operator unseal TxGdasK3Wt5ptBw/YHOmE4ZfnyICVusP2eK0NBk9MDRX / $ vault operator unseal +VVeL1qtFH1L3euMTocIPrD28PbhtlLIrpyDO25w1aW+ / $ vault operator unseal 9YEh9Lq0Pgc5TU4EMGTjV2o+88vgC/azrc+vm5GOz3UY ``` * 這時再去看 container 狀態就會是準備好的 ``` $ kubectl -n vault get po NAME READY STATUS RESTARTS AGE vault-0 1/1 Running 0 108m vault-agent-injector-556c5dd8fb-x9r2n 1/1 Running 0 108m ``` ## 配置 Vault * 進到 Vault pod,並且登入 vault ``` $ kubectl exec --stdin=true --tty=true vault-0 -n vault -- /bin/sh $ vault login <你的 Root Token> ``` * 啟用 Secret Engine,路徑為 `secret/` ``` $ vault secrets enable -path=secret kv-v2 $ vault secrets list Path Type Accessor Description ---- ---- -------- ----------- cubbyhole/ cubbyhole cubbyhole_de79ebce per-token private secret storage identity/ identity identity_46e6dfb6 identity store secret/ kv kv_fa60edbf n/a sys/ system system_e575a6fa system endpoints used for control, policy and debugging ``` * 建立並套用策略,我們將建立一項只允許讀取 path 路徑下 `secret*` 所有的策略 ``` $ cat <<EOF > /home/vault/read-policy.hcl path "secret*" { capabilities = ["read"] } EOF ``` > path 指的是 Vault 的 API 路徑(也就是 secret engine mount point 下的資源路徑) * 套用策略 ``` $ vault policy write read-policy /home/vault/read-policy.hcl ``` * 啟用 Kubernetes 身份驗證 ``` $ vault auth enable kubernetes ``` * 設定 vault 和 Kubernetes 身份驗證 ``` $ vault write auth/kubernetes/config \ token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \ kubernetes_host=https://${KUBERNETES_PORT_443_TCP_ADDR}:443 \ kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt ``` * 在 vaule 建立一個 Role 名稱叫 `vault-role`,將上述政策綁定到 `myapp`、`myapp2` namespace 的 Kubernetes sa(vault-serviceaccount)。這讓 sa 能夠存取存放在 Vault 中的 secret ``` $ vault write auth/kubernetes/role/vault-role \ bound_service_account_names=vault-serviceaccount \ bound_service_account_namespaces=myapp,myapp2 \ policies=read-policy \ audience=https://kubernetes.default.svc.cluster.local \ ttl=1h ``` > audience : 會去驗證啟動的 pod 的 token 是否有帶這個 aud,預設 k8s 做出來的 pod token 都是帶 `https://kubernetes.default.svc.cluster.local` * 驗證 ``` # 自己創建一個 nginx pod $ kubectl -n myapp2 exec vault-test-c487c5c9d-s6wfq -- sh -c "cat /var/run/secrets/kubernetes.io/serviceaccount/token | cut -d '.' -f 2 | base64 -d" Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init) {"aud":["https://kubernetes.default.svc.cluster.local"],"exp":1796114554,"iat":1764578554,"iss":"https://kubernetes.default.svc.cluster.local","jti":"5de198a9-9fad-47bc-836c-2460b1c5e6c0","kubernetes.io":{"namespace":"myapp2","node":{"name":"w1","uid":"8efd2264-e12e-4ae8-ba3f-ab7841163d25"},"pod":{"name":"vault-test-c487c5c9d-s6wfq","uid":"2be8d886-aa8a-4a6b-b80c-29c0832b44c7"},"serviceaccount":{"name":"vault-serviceaccount","uid":"0293e354-bd11-40b2-b487-d0dd64b12bbe"},"warnafter":1764582161},"nbf":1764578554,"sub":"system:serviceaccount:myapp2:vault-serviceaccount"} ``` ## 在 vault 創建 secret * 可以使用 Vault CLI 和 Vault UI 創建 * 以下使用指令建立一個 secret 名叫 `my-first-secret`,內容包含 `username=bigred`、`password='P@ssw0rd123'` ,並將這個 secret 寫入到 vault 中 ``` $ vault kv put secret/my-first-secret username=bigred password='P@ssw0rd123' ``` * 檢視產生的 secret ``` $ vault kv list secret Keys ---- my-first-secret ``` * 查看 secret 內容 ``` $ vault kv get -mount="secret" "my-first-secret" ======= Secret Path ======= secret/data/my-first-secret ======= Metadata ======= Key Value --- ----- created_time 2025-12-01T08:26:10.403266484Z custom_metadata <nil> deletion_time n/a destroyed false version 1 ====== Data ====== Key Value --- ----- password P@ssw0rd123 username bigred # 退出 vault pod $ exit ``` ## 在 pod 內存取 vault 中的 secret * 我們前面創建了一個 vault-role 讓 sa(vault-serviceaccount)可以存取放在 vault 中的 secret,但此時還沒在 k8s 中創建 sa(vault-serviceaccount),因此我們要先創建他。 ``` $ kubectl create ns myapp $ kubectl create ns myapp2 $ kubectl -n myapp create sa vault-serviceaccount $ kubectl -n myapp2 create sa vault-serviceaccount ``` * 創建 nginx pod ``` $ nano vault-secret-test-deploy.yaml apiVersion: apps/v1 kind: Deployment metadata: name: vault-test labels: app: read-vault-secret spec: selector: matchLabels: app: read-vault-secret replicas: 1 template: metadata: annotations: vault.hashicorp.com/agent-inject: "true" # 啟用 Vault Agent 注入 sidcar 到 pod 內 vault.hashicorp.com/agent-inject-status: "update" # 確保 secret 更新後也會自動更新到 pod 內 vault.hashicorp.com/agent-inject-secret-my-first-secret: "secret/my-first-secret" # 定義從 vault 拿到的 secret 的「來源」與「檔名」 # 定義「檔案內容格式」 vault.hashicorp.com/agent-inject-template-my-first-secret: | {{- with secret "secret/my-first-secret" -}} username={{ .Data.data.username }} password={{ .Data.data.password }} {{- end }} vault.hashicorp.com/role: "vault-role" # 用 vault-role 登入 Vault vault.hashicorp.com/template-static-secret-render-interval: "10s" # 每十秒跟 vault 同步 secret 內容,預設是 5m labels: app: read-vault-secret spec: serviceAccountName: vault-serviceaccount containers: - name: nginx image: nginx ``` ``` $ kubectl -n myapp apply -f vault-secret-test-deploy.yaml $ kubectl -n myapp2 apply -f vault-secret-test-deploy.yaml ``` * 可以看到我們創建的 workload 都有 sidecar,這個 sidecar 就是 vault 注入的 container,用來同步儲存在 vault 裡的 secret 資訊。 ``` $ kubectl -n myapp get pod NAME READY STATUS RESTARTS AGE vault-test-c487c5c9d-r84kc 2/2 Running 0 86s $ kubectl -n myapp2 get pod NAME READY STATUS RESTARTS AGE vault-test-c487c5c9d-s6wfq 2/2 Running 0 6s ``` * 檢視 `myapp` 和 `myapp2` namespace 下的 workload 都可以看到 vault 創建的 secret 內容,並且是放在 pod 的 `/vault/secrets/my-first-secret` 檔案內。 ``` $ kubectl -n myapp exec vault-test-c487c5c9d-r84kc -- sh -c "cat /vault/secrets/my-first-secret" Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init) username=bigred password=P@ssw0rd123 $ kubectl -n myapp2 exec vault-test-c487c5c9d-s6wfq -- sh -c "cat /vault/secrets/my-first-secret" Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init) username=bigred password=P@ssw0rd123 ``` ## 動態更新 secret * 進到 vault 內,手動更新 secret 內容 ``` $ kubectl exec --stdin=true --tty=true vault-0 -n vault -- /bin/sh $ vault kv put secret/my-first-secret username="haha" password="haha" $ exit ``` * 確認 pod 都已更新 secret 內容 ``` $ kubectl -n myapp exec vault-test-c487c5c9d-r84kc -- sh -c "cat /vault/secrets/my-first-secret" Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init) username=haha password=haha $ kubectl -n myapp exec vault-test-c487c5c9d-r84kc -- sh -c "cat /vault/secrets/my-first-secret" Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init) username=haha password=haha ``` ## 參考 https://developer.hashicorp.com/vault/docs/deploy/kubernetes/injector/annotations https://developer.hashicorp.com/vault/tutorials/kubernetes/vault-secrets-operator https://medium.com/@muppedaanvesh/a-hand-on-guide-to-vault-in-kubernetes-%EF%B8%8F-1daf73f331bd