# 實作 ETCD 資料加密

## CIS Kubernetes Benchmark 掃描
* 透過 CIS Kubernetes Benchmark 掃描時發現有存在以下漏洞，就是 etcd 在儲存 secret 物件時，是以 key-value 的方式儲存，而這些敏感資料都是明碼可以直接看到的資料。

CIS Kubernetes Benchmark v1.10 – 1.2.271.2.27 Ensure that the --encryption-provider-config argument is set as appropriate (Manual)
Profile Applicability:
• Level 1 - Master Node
Description:
Encrypt etcd key-value store.
Rationale:
etcd is a highly available key-value store used by Kubernetes deployments for persistent storage of all of its REST API objects. These objects are sensitive in nature and should be encrypted at rest to avoid any disclosures.

## 實作
* 安裝 etcdctl

$ ETCD_RELEASE=$(curl -s https://api.github.com/repos/etcd-io/etcd/releases/latest|grep tag_name | cut -d '"' -f 4)

$ wget https://github.com/etcd-io/etcd/releases/download/${ETCD_RELEASE}/etcd-${ETCD_RELEASE}-linux-amd64.tar.gz

$ tar zxvf etcd-${ETCD_RELEASE}-linux-amd64.tar.gz

$ sudo cp -rp etcd-${ETCD_RELEASE}-linux-amd64/etcdctl /usr/local/bin

$ IP=$(kubectl get node single-control-plane1 -o jsonpath="{.status.addresses[?(@.type=='InternalIP')].address}")

### 在 kind 環境實作
* 設定與測試 etcdctl

$ alias etcdctl="ETCDCTL_API=3 sudo /usr/local/bin/etcdctl \
  --endpoints=${IP}:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/apiserver-etcd-client.crt \
  --key=/etc/kubernetes/pki/apiserver-etcd-client.key"

# 測試建立一個 secret
$ kubectl create secret generic demo-secret \
  --from-literal=username=root \
  --from-literal=password=rootpass

# 直接從 etcd 查看資料是以明碼的方式
$ etcdctl get /registry/secrets/default/demo-secret
...
?{"f:data":{".":{},"f:password":{},"f:username":{}},"f:type":{}}B
passworrootpass
usernameroot␦Opaque␦"

## 建立加密設定檔案
* 針對 secret 物件做加密

$ sudo mkdir /etc/kubernetes/enc

$ cat <<EOF | sudo tee /etc/kubernetes/enc/enc.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          keys:
            - name: key1
              secret: $(head -c 32 /dev/urandom | base64)
      - identity: {}
EOF

* 更新 API Server 的設定檔

$ nano /etc/kubernetes/manifests/kube-apiserver.yaml
...
spec:
  containers:
  - command:
    - kube-apiserver
    ...
    - --encryption-provider-config=/etc/kubernetes/enc/enc.yaml # add this line
...
 volumeMounts:
    ...
    - name: enc                           # add this line
      mountPath: /etc/kubernetes/enc      # add this line
      readOnly: true                      # add this line
...
volumes:
  ...
  - name: enc                             # add this line
    hostPath:                             # add this line
      path: /etc/kubernetes/enc           # add this line
      type: DirectoryOrCreate             # add this line
  ...

* 重新產生 secret，確認資料都已加密

$ kubectl get secrets --all-namespaces -o json | kubectl replace -f -

$ etcdctl get /registry/secrets/default/demo-secret
/registry/secrets/default/demo-secret
k8s:enc:aescbc:v1:key1:�2P�� #G��:�     ��
���B�l{�T�sS�t���ѩd1�2��;�߇��,P^\�5K���␦M*Q�ݱԜ*���=y���7D�r���y.��M:y�T�3�tv��m9�Kս��y|٨�1.�τ8X��Þ���GI%�Vڋ�>�G�;B�v(>J�2-�J�T��nu�
    �ce
       �#+g}��  �׉��Sb@iRxˈ�c
�6�Xs�L-5�0��'�v)�N�         ;F��<�x-���ꇖп�6��]@�#�� �����S20�`���
                    5�q<"n�nһ��UEEU��ڧ�$o}0�ӥB2�Za�d/���T,Uɇ��3�L�+vQk�U���mc_=��|��-|��^(�{�Oȧݣ��ؽ,~"�~%e

### RKE2 環境預設已啟用 etcd 資料加密的功能

$ cat /var/lib/rancher/rke2/agent/pod-manifests/kube-apiserver.yaml | grep encryption-provider-config
    - --encryption-provider-config=/var/lib/rancher/rke2/server/cred/encryption-config.json
    
$ cat /var/lib/rancher/rke2/server/cred/encryption-config.json | jq .
{
  "kind": "EncryptionConfiguration",
  "apiVersion": "apiserver.config.k8s.io/v1",
  "resources": [
    {
      "resources": [
        "secrets"
      ],
      "providers": [
        {
          "aescbc": {
            "keys": [
              {
                "name": "aescbckey",
                "secret": "38OJP53EAGjwxFy//MJ3yaSO6OO7JSKDHKFZAWHDqs0="
              }
            ]
          }
        },
        {
          "identity": {}
        }
      ]
    }
  ]
}