# HashiCorp Vault Secret Protect
## Vault 使用背景
Kubernetes 原生 Secret 機制預設僅採用 Base64 編碼,這僅是一種數據格式轉換而非加密技術。這意味著任何擁有 Namespace 讀取權限的人員或程序,都能輕易將其還原為明文,缺乏足夠強度的資安防護機制。
而透過 Vault 的 Agent Sidecar Injector 功能,可以將 secret 內容儲存在 Vault 內,透過 Vault 的保護機制來確保 secret 不被竊取。
## 安裝 Vault
```
$ helm repo add hashicorp https://helm.releases.hashicorp.com
$ helm search repo hashicorp/vault
NAME CHART VERSION APP VERSION DESCRIPTION
hashicorp/vault 0.31.0 1.20.4 Official HashiCorp Vault Chart
hashicorp/vault-secrets-gateway 0.0.2 0.1.0 A Helm chart for Kubernetes
hashicorp/vault-secrets-operator 1.0.1 1.0.1 Official Vault Secrets Operator Chart
```
* K8s 環境已準備好 `nfs-csi` csi,或是使用自己環境的也可以。
```
$ kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-csi nfs.csi.k8s.io Retain Immediate false 35d
```
* 安裝 vault 設定資料永存,這裡需要指定自己環境的 csi
```
$ helm install vault hashicorp/vault \
--create-namespace \
--set='ui.enabled=true' \
--set='ui.serviceType=NodePort' \
--set='server.dataStorage.storageClass=nfs-csi' \
--set='server.dataStorage.accessMode=ReadWriteMany' \
--set='server.dataStorage.size=20Gi' \
--namespace vault
```
* 這時 vault pod 的 container 是還沒有準備好的,我們還需要手動初始化他
```
$ kubectl -n vault get all
NAME READY STATUS RESTARTS AGE
pod/vault-0 0/1 Running 0 10s
pod/vault-agent-injector-556c5dd8fb-8g4bd 1/1 Running 0 11s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/vault ClusterIP 10.96.33.246 <none> 8200/TCP,8201/TCP 11s
service/vault-agent-injector-svc ClusterIP 10.96.243.132 <none> 443/TCP 11s
service/vault-internal ClusterIP None <none> 8200/TCP,8201/TCP 11s
service/vault-ui NodePort 10.96.1.24 <none> 8200:30768/TCP 11s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/vault-agent-injector 1/1 1 1 11s
NAME DESIRED CURRENT READY AGE
replicaset.apps/vault-agent-injector-556c5dd8fb 1 1 1 11s
NAME READY AGE
statefulset.apps/vault 0/1 11s
```
* 初始化 vault,這邊的 Unseal Key 需要自己保留,如果 vault pod 重啟,需要透過以下的 key 來解鎖。
```
$ kubectl exec -ti vault-0 -n vault -- /bin/sh
/ $ vault operator init
Unseal Key 1: TxGdasK3Wt5ptBw/YHOmE4ZfnyICVusP2eK0NBk9MDRX
Unseal Key 2: +VVeL1qtFH1L3euMTocIPrD28PbhtlLIrpyDO25w1aW+
Unseal Key 3: 9YEh9Lq0Pgc5TU4EMGTjV2o+88vgC/azrc+vm5GOz3UY
Unseal Key 4: C2MZTuwxxHaa3cAcFi7BIAd41D/JQz4BSVDu42k+kz3y
Unseal Key 5: zYUxWqhDz96nU5iO7pYkS1jRndfU3fBq319v0QKv/sNL
Initial Root Token: hvs.lmGfDNMDgpl6uxb39exPTg4N
Vault initialized with 5 key shares and a key threshold of 3. Please securely
distribute the key shares printed above. When the Vault is re-sealed,
restarted, or stopped, you must supply at least 3 of these keys to unseal it
before it can start servicing requests.
Vault does not store the generated root key. Without at least 3 keys to
reconstruct the root key, Vault will remain permanently sealed!
It is possible to generate new unseal keys, provided you have a quorum of
existing unseal keys shares. See "vault operator rekey" for more information.
# 輸入其中的三把 key 用來解鎖 Shamir's Secret Sharing 的演算法
/ $ vault operator unseal TxGdasK3Wt5ptBw/YHOmE4ZfnyICVusP2eK0NBk9MDRX
/ $ vault operator unseal +VVeL1qtFH1L3euMTocIPrD28PbhtlLIrpyDO25w1aW+
/ $ vault operator unseal 9YEh9Lq0Pgc5TU4EMGTjV2o+88vgC/azrc+vm5GOz3UY
```
* 這時再去看 container 狀態就會是準備好的
```
$ kubectl -n vault get po
NAME READY STATUS RESTARTS AGE
vault-0 1/1 Running 0 108m
vault-agent-injector-556c5dd8fb-x9r2n 1/1 Running 0 108m
```
## 配置 Vault
* 進到 Vault pod,並且登入 vault
```
$ kubectl exec --stdin=true --tty=true vault-0 -n vault -- /bin/sh
$ vault login <你的 Root Token>
```
* 啟用 Secret Engine,路徑為 `secret/`
```
$ vault secrets enable -path=secret kv-v2
$ vault secrets list
Path Type Accessor Description
---- ---- -------- -----------
cubbyhole/ cubbyhole cubbyhole_de79ebce per-token private secret storage
identity/ identity identity_46e6dfb6 identity store
secret/ kv kv_fa60edbf n/a
sys/ system system_e575a6fa system endpoints used for control, policy and debugging
```
* 建立並套用策略,我們將建立一項只允許讀取 path 路徑下 `secret*` 所有的策略
```
$ cat <<EOF > /home/vault/read-policy.hcl
path "secret*" {
capabilities = ["read"]
}
EOF
```
> path 指的是 Vault 的 API 路徑(也就是 secret engine mount point 下的資源路徑)
* 套用策略
```
$ vault policy write read-policy /home/vault/read-policy.hcl
```
* 啟用 Kubernetes 身份驗證
```
$ vault auth enable kubernetes
```
* 設定 vault 和 Kubernetes 身份驗證
```
$ vault write auth/kubernetes/config \
token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
kubernetes_host=https://${KUBERNETES_PORT_443_TCP_ADDR}:443 \
kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
```
* 在 vaule 建立一個 Role 名稱叫 `vault-role`,將上述政策綁定到 `myapp`、`myapp2` namespace 的 Kubernetes sa(vault-serviceaccount)。這讓 sa 能夠存取存放在 Vault 中的 secret
```
$ vault write auth/kubernetes/role/vault-role \
bound_service_account_names=vault-serviceaccount \
bound_service_account_namespaces=myapp,myapp2 \
policies=read-policy \
audience=https://kubernetes.default.svc.cluster.local \
ttl=1h
```
> audience : 會去驗證啟動的 pod 的 token 是否有帶這個 aud,預設 k8s 做出來的 pod token 都是帶 `https://kubernetes.default.svc.cluster.local`
* 驗證
```
# 自己創建一個 nginx pod
$ kubectl -n myapp2 exec vault-test-c487c5c9d-s6wfq -- sh -c "cat /var/run/secrets/kubernetes.io/serviceaccount/token | cut -d '.' -f 2 | base64 -d"
Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init)
{"aud":["https://kubernetes.default.svc.cluster.local"],"exp":1796114554,"iat":1764578554,"iss":"https://kubernetes.default.svc.cluster.local","jti":"5de198a9-9fad-47bc-836c-2460b1c5e6c0","kubernetes.io":{"namespace":"myapp2","node":{"name":"w1","uid":"8efd2264-e12e-4ae8-ba3f-ab7841163d25"},"pod":{"name":"vault-test-c487c5c9d-s6wfq","uid":"2be8d886-aa8a-4a6b-b80c-29c0832b44c7"},"serviceaccount":{"name":"vault-serviceaccount","uid":"0293e354-bd11-40b2-b487-d0dd64b12bbe"},"warnafter":1764582161},"nbf":1764578554,"sub":"system:serviceaccount:myapp2:vault-serviceaccount"}
```
## 在 vault 創建 secret
* 可以使用 Vault CLI 和 Vault UI 創建
* 以下使用指令建立一個 secret 名叫 `my-first-secret`,內容包含 `username=bigred`、`password='P@ssw0rd123'` ,並將這個 secret 寫入到 vault 中
```
$ vault kv put secret/my-first-secret username=bigred password='P@ssw0rd123'
```
* 檢視產生的 secret
```
$ vault kv list secret
Keys
----
my-first-secret
```
* 查看 secret 內容
```
$ vault kv get -mount="secret" "my-first-secret"
======= Secret Path =======
secret/data/my-first-secret
======= Metadata =======
Key Value
--- -----
created_time 2025-12-01T08:26:10.403266484Z
custom_metadata <nil>
deletion_time n/a
destroyed false
version 1
====== Data ======
Key Value
--- -----
password P@ssw0rd123
username bigred
# 退出 vault pod
$ exit
```
## 在 pod 內存取 vault 中的 secret
* 我們前面創建了一個 vault-role 讓 sa(vault-serviceaccount)可以存取放在 vault 中的 secret,但此時還沒在 k8s 中創建 sa(vault-serviceaccount),因此我們要先創建他。
```
$ kubectl create ns myapp
$ kubectl create ns myapp2
$ kubectl -n myapp create sa vault-serviceaccount
$ kubectl -n myapp2 create sa vault-serviceaccount
```
* 創建 nginx pod
```
$ nano vault-secret-test-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: vault-test
labels:
app: read-vault-secret
spec:
selector:
matchLabels:
app: read-vault-secret
replicas: 1
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true" # 啟用 Vault Agent 注入 sidcar 到 pod 內
vault.hashicorp.com/agent-inject-status: "update" # 確保 secret 更新後也會自動更新到 pod 內
vault.hashicorp.com/agent-inject-secret-my-first-secret: "secret/my-first-secret" # 定義從 vault 拿到的 secret 的「來源」與「檔名」
# 定義「檔案內容格式」
vault.hashicorp.com/agent-inject-template-my-first-secret: |
{{- with secret "secret/my-first-secret" -}}
username={{ .Data.data.username }}
password={{ .Data.data.password }}
{{- end }}
vault.hashicorp.com/role: "vault-role" # 用 vault-role 登入 Vault
vault.hashicorp.com/template-static-secret-render-interval: "10s" # 每十秒跟 vault 同步 secret 內容,預設是 5m
labels:
app: read-vault-secret
spec:
serviceAccountName: vault-serviceaccount
containers:
- name: nginx
image: nginx
```
```
$ kubectl -n myapp apply -f vault-secret-test-deploy.yaml
$ kubectl -n myapp2 apply -f vault-secret-test-deploy.yaml
```
* 可以看到我們創建的 workload 都有 sidecar,這個 sidecar 就是 vault 注入的 container,用來同步儲存在 vault 裡的 secret 資訊。
```
$ kubectl -n myapp get pod
NAME READY STATUS RESTARTS AGE
vault-test-c487c5c9d-r84kc 2/2 Running 0 86s
$ kubectl -n myapp2 get pod
NAME READY STATUS RESTARTS AGE
vault-test-c487c5c9d-s6wfq 2/2 Running 0 6s
```
* 檢視 `myapp` 和 `myapp2` namespace 下的 workload 都可以看到 vault 創建的 secret 內容,並且是放在 pod 的 `/vault/secrets/my-first-secret` 檔案內。
```
$ kubectl -n myapp exec vault-test-c487c5c9d-r84kc -- sh -c "cat /vault/secrets/my-first-secret"
Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init)
username=bigred
password=P@ssw0rd123
$ kubectl -n myapp2 exec vault-test-c487c5c9d-s6wfq -- sh -c "cat /vault/secrets/my-first-secret"
Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init)
username=bigred
password=P@ssw0rd123
```
## 動態更新 secret
* 進到 vault 內,手動更新 secret 內容
```
$ kubectl exec --stdin=true --tty=true vault-0 -n vault -- /bin/sh
$ vault kv put secret/my-first-secret username="haha" password="haha"
$ exit
```
* 確認 pod 都已更新 secret 內容
```
$ kubectl -n myapp exec vault-test-c487c5c9d-r84kc -- sh -c "cat /vault/secrets/my-first-secret"
Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init)
username=haha
password=haha
$ kubectl -n myapp exec vault-test-c487c5c9d-r84kc -- sh -c "cat /vault/secrets/my-first-secret"
Defaulted container "nginx" out of: nginx, vault-agent, vault-agent-init (init)
username=haha
password=haha
```
## 參考
https://developer.hashicorp.com/vault/docs/deploy/kubernetes/injector/annotations
https://developer.hashicorp.com/vault/tutorials/kubernetes/vault-secrets-operator
https://medium.com/@muppedaanvesh/a-hand-on-guide-to-vault-in-kubernetes-%EF%B8%8F-1daf73f331bd