Kubernetes CLI plugin Manager- Krew

--- tags: Kubernetes, Plugin Manager description: Kubernetes Plugin Manager robots: index, follow --- # Kubernetes CLI plugin Manager- Krew krew是一個kubectl的延伸套件管理工具，可以很快速利用他安裝很多指令工具，雖然他不像Web UI有很華麗的操作方式，不過在指令輸出上是個可圈可點的管理工具。 ## 1. Environment 1. Cluster Manager: Rancher 2. Kubernetes: 1.31.7 3. OS: SUSE Linux Enterprise Server (SLES), openSUSE Leap或其他OS也可以。 4. Master * 3, Worker * 3 5. 有安裝git ## 2. install Copy 1-9行直接在系統上執行 ```shell= ( set -x; cd "$(mktemp -d)" && OS="$(uname | tr '[:upper:]' '[:lower:]')" && ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/$arm$$64$\?.*/\1\2/' -e 's/aarch64$/arm64/')" && KREW="krew-${OS}_${ARCH}" && curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" && tar zxvf "${KREW}.tar.gz" && ./"${KREW}" install krew ) ++ mktemp -d + cd /tmp/tmp.ezIe53LMlR ++ uname ++ tr '[:upper:]' '[:lower:]' + OS=linux ++ uname -m ++ sed -e s/x86_64/amd64/ -e 's/$arm$$64$\?.*/\1\2/' -e 's/aarch64$/arm64/' + ARCH=amd64 + KREW=krew-linux_amd64 + curl -fsSLO https://github.com/kubernetes-sigs/krew/releases/latest/download/krew-linux_amd64.tar.gz + tar zxvf krew-linux_amd64.tar.gz ./._LICENSE tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance' ./LICENSE ./._krew-linux_amd64 tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance' ./krew-linux_amd64 + ./krew-linux_amd64 install krew Adding "default" plugin index from https://github.com/kubernetes-sigs/krew-index.git. Updated the local copy of plugin index. Installing plugin: krew Installed plugin: krew \ | Use this plugin: | kubectl krew | Documentation: | https://krew.sigs.k8s.io/ | Caveats: | \ | | krew is now installed! To start using kubectl plugins, you need to add | | krew's installation directory to your PATH: | | | | * macOS/Linux: | | - Add the following to your ~/.bashrc or ~/.zshrc: | | export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH" | | - Restart your shell. | | | | * Windows: Add %USERPROFILE%\.krew\bin to your PATH environment variable | | | | To list krew commands and to get help, run: | | $ kubectl krew | | For a full list of available plugins, run: | | $ kubectl krew search | | | | You can find documentation at | | https://krew.sigs.k8s.io/docs/user-guide/quickstart/. | / / ``` 將以下內容加入環境參數最後一行(~/.bashrc) ```shell= export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH" ``` 重新載入環境參數檔 ```shell= demo-m1:~# source ~/.bashrc ``` 測試一下，確認有安裝成功 ```shell= demo-m1:~# kubectl krew krew is the kubectl plugin manager. You can invoke krew through kubectl: "kubectl krew [command]..." Usage: kubectl krew [command] ... ... ... ``` ## 3. check plugins available list ```shell= demo-m1:~# kubectl krew search NAME DESCRIPTION INSTALLED access-matrix Show an RBAC access matrix for server resources no advise-psp Suggests PodSecurityPolicies for cluster. no allctx Run commands on contexts in your kubeconfig no apparmor-manager Manage AppArmor profiles for cluster. no assert Assert Kubernetes resources no auth-proxy Authentication proxy to a pod or service no azad-proxy Generate and handle authentication for azad-kub... no bd-xray Run Black Duck Image Scans no ... ... ... ``` ## 4. install plugin 安裝三個套件來試試： 1. pod-lens 可以透過指令呈現資源間的關係，加上顏色輸出效果更好了。 ```shell= demo-m1:~# kubectl krew install pod-lens Updated the local copy of plugin index. Installing plugin: pod-lens Installed plugin: pod-lens \ | Use this plugin: | kubectl pod-lens | Documentation: | https://pod-lens.guoxudong.io / WARNING: You installed plugin "pod-lens" from the krew-index plugin repository. These plugins are not audited for security by the Krew maintainers. Run them at your own risk. ``` 2. kubesec-scan 可以針對資源進行security context掃描，掃瞄完畢後會有建議的動作。 ```shell= demo-m1:~# kubectl krew install kubesec-scan Updated the local copy of plugin index. Installing plugin: kubesec-scan Installed plugin: kubesec-scan \ | Use this plugin: | kubectl kubesec-scan | Documentation: | https://github.com/controlplaneio/kubectl-kubesec / WARNING: You installed plugin "kubesec-scan" from the krew-index plugin repository. These plugins are not audited for security by the Krew maintainers. Run them at your own risk. ``` 3. cyclonus 使用表格式呈現network policy的關係，比自己閱讀yaml的方式好非常多。 ```shell= demo-m1:~# kubectl krew install cyclonus Updated the local copy of plugin index. Installing plugin: cyclonus Installed plugin: cyclonus \ | Use this plugin: | kubectl cyclonus | Documentation: | https://github.com/mattfenwick/kubectl-cyclonus / WARNING: You installed plugin "cyclonus" from the krew-index plugin repository. These plugins are not audited for security by the Krew maintainers. Run them at your own risk. ``` ### Pod-Lens 看一下default namespace所有的東西，但是各資源關聯透過指令看的時候，要多仔細對一下。 ```shell= demo-m1:~# kubectl get all -n museum-dino-kiosk |head -n 10 NAME READY STATUS RESTARTS AGE pod/ai-sim-service-69bb94c5bf-2nr6d 1/1 Running 0 45m pod/build-a-dino-67bd9798b7-q6nrv 1/1 Running 0 45m pod/kiosk-visitors-677fb7444d-t6jmh 1/1 Running 0 45m pod/kiosk-web-8664bd9b66-bswts 1/1 Running 0 45m pod/printer-3d-65497db7b-lk68b 1/1 Running 0 45m pod/printing-queue-59b646fc77-ksf2k 1/1 Running 0 45m pod/printing-service-8645b7685d-xvs5k 1/1 Running 0 45m pod/shipping-76cbbdc6f4-ftgwc 1/1 Running 0 45m ``` 透過pod-lens可以使用箭號選擇要看哪一個pod（本例使用testlens-7f9b6f66b5-hcrtq）。 ```shell= demo-m1:~# kubectl pod-lens -n museum-dino-kiosk kubectl pod-lens -n museum-dino-kiosk Use the arrow keys to navigate: ↓ ↑ → ← Select Pod ▸ ai-sim-service-69bb94c5bf-2nr6d build-a-dino-67bd9798b7-q6nrv kiosk-visitors-677fb7444d-t6jmh kiosk-web-8664bd9b66-bswts ↓ printer-3d-65497db7b-lk68b --------- Info ---------- Namespace: museum-dino-kiosk Node: demo-w2 Status: Running Pod IP: 10.42.4.177 ``` 選擇之後看輸出，顯示這個pod跟其他資源的樹狀關聯與pod IP。 ```shell= demo-m1:~$ ✔ ai-sim-service-69bb94c5bf-2nr6d [Namespace] museum-dino-kiosk └─┬ [Namespace] museum-dino-kiosk Replica: 1/1 └─┬ [Deployment] ai-sim-service [Ready] Node IP: 10.107.88.238 ├─┬ [Node] demo-w2 [Running] Pod IP: 10.42.4.177 │ └─┬ [Pod] ai-sim-service-69bb94c5bf-2nr6d [Running] Restart: 0 │ └── [Container] simservice ├── [ConfigMap] simservice-cm └── [ConfigMap] kube-root-ca.crt Related Resources Kind: Deployment Name: ai-sim-service Replicas: 1 --- --- Kind: Deployment Name: build-a-dino Replicas: 1 ``` ![截圖 2025-04-18 11.07.36](https://hackmd.io/_uploads/SyF7fB1yeg.png) :::info 1. 可以透過指令與輸出的樹狀結構，從指令的方式加強理解資源間的關係。 2. 從指令輸出有添加顏色，強化可讀性。 ::: ### kubesec-scan 檢查資源的SecurityContext是否達到建議狀態，並且列出建議。 ```shell= demo-m1:~# kubectl kubesec-scan -n museum-dino-kiosk deployment shipping scanning deployment shipping in namespace museum-dino-kiosk kubesec.io score: 4 ----------------- Advise1. .metadata .annotations ."container.apparmor.security.beta.kubernetes.io/nginx" Well defined AppArmor policies may provide greater protection from unknown threats. WARNING: NOT PRODUCTION READY 2. .spec .serviceAccountName Service accounts restrict Kubernetes API access and should be configured with least privilege 3. .metadata .annotations ."container.seccomp.security.alpha.kubernetes.io/pod" Seccomp profiles set minimum privilege and secure against unknown threats 4. .spec .automountServiceAccountToken == false Disabling the automounting of Service Account Token reduces the attack surface of the API server 5. .spec, .spec.containers[] | .securityContext .runAsGroup -gt 10000 Run as a high-UID group to avoid conflicts with the host's groups 6. .spec, .spec.containers[] | .securityContext .runAsNonRoot == true Force the running image to run as a non-root user to ensure least privilege 7. .spec, .spec.containers[] | .securityContext .runAsUser -gt 10000 Run as a high-UID user to avoid conflicts with the host's users 8. containers[] .securityContext .capabilities .drop Reducing kernel capabilities available to a container limits its attack surface 9. containers[] .securityContext .capabilities .drop | index("ALL") Drop all capabilities and add only those required to reduce syscall attack surface 10. containers[] .securityContext .readOnlyRootFilesystem == true An immutable root filesystem can prevent malicious binaries being added to PATH and increase attack cost ``` :::info 1. 掃描的結果需要確認是否為該程式應用的一環。 2. 過與不及的安全性設定都會帶來更多的問題，需要恰如其分的設置。 ::: ### cyclonus 建立網頁服務 ```shell= demo-m1:~ # kubectl create deploy web --image=registry.suse.com/suse/nginx:latest --port=80 deployment.apps/web created demo-m1:~ # kubectl expose deployment web --target-port 80 --port 80 --type NodePort service/web exposed ``` 確認環境狀態 ```shell= demo-m1:~ # kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 47h web NodePort 10.43.109.212 <none> 80:32348/TCP 25s demo-m1:~ # kubectl get no -owide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME demo-m1 Ready control-plane,etcd,master 47h v1.31.7+rke2r1 10.107.88.234 <none> SUSE Linux Enterprise Server 15 SP6 6.4.0-150600.21-default containerd://2.0.4-k3s2 demo-m2 Ready control-plane,etcd,master 47h v1.31.7+rke2r1 10.107.88.235 <none> SUSE Linux Enterprise Server 15 SP6 6.4.0-150600.21-default containerd://2.0.4-k3s2 demo-m3 Ready control-plane,etcd,master 47h v1.31.7+rke2r1 10.107.88.236 <none> SUSE Linux Enterprise Server 15 SP6 6.4.0-150600.21-default containerd://2.0.4-k3s2 demo-w1 Ready worker 46h v1.31.7+rke2r1 10.107.88.237 <none> SUSE Linux Enterprise Server 15 SP6 6.4.0-150600.21-default containerd://2.0.4-k3s2 demo-w2 Ready worker 46h v1.31.7+rke2r1 10.107.88.238 <none> SUSE Linux Enterprise Server 15 SP6 6.4.0-150600.21-default containerd://2.0.4-k3s2 demo-w3 Ready worker 46h v1.31.7+rke2r1 10.107.88.239 <none> SUSE Linux Enterprise Server 15 SP6 6.4.0-150600.21-default containerd://2.0.4-k3s2 demo-m1:~ # curl 10.107.88.239:32348 |head -n 4 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 366 100 366 0 0 105k 0 --:--:-- --:--:-- --:--:-- 119k <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> ``` 建立一個network policy拒絕所有往default namespace所有的連線。 **Yaml File: np.yaml** ```yaml= apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: denyalltodefault namespace: default spec: podSelector: {} policyTypes: - Egress - Ingress ``` 建立network policy ```shell= demo-m1:~ # kubectl create -f np.yaml networkpolicy.networking.k8s.io/denyalltodefault created ``` 確認是否可連線（不行）。 ```shell= demo-m1:~ # curl 10.107.88.239:32348 |head -n 4 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0^C ``` 透過cyclonus檢查network policy(Ingress, Egress全部無法連通) ```shell= demo-m1:~ # kubectl cyclonus --mode explain -n default explained policies: +---------+--------------------+--------------------------+-----------------+------------------------+ | TYPE | TARGET | SOURCE RULES | PEER | PORT/PROTOCOL | +---------+--------------------+--------------------------+-----------------+------------------------+ | Ingress | namespace: default | default/denyalltodefault | no pods, no ips | no ports, no protocols | | | all pods | | | | +---------+--------------------+--------------------------+-----------------+------------------------+ | | | | | | +---------+--------------------+--------------------------+-----------------+------------------------+ | Egress | namespace: default | default/denyalltodefault | no pods, no ips | no ports, no protocols | | | all pods | | | | +---------+--------------------+--------------------------+-----------------+------------------------+ ``` 刪除network policy、確認連線是否正常。 ```shell= demo-m1:~ # kubectl delete -f np.yaml networkpolicy.networking.k8s.io "denyalltodefault" deleted demo-m1:~ # kubectl cyclonus --mode explain -n default explained policies: +------+--------+--------------+------+---------------+ | TYPE | TARGET | SOURCE RULES | PEER | PORT/PROTOCOL | +------+--------+--------------+------+---------------+ | | | | | | +------+--------+--------------+------+---------------+ demo-m1:~ # curl 10.107.88.239:32348 |head -n 4 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 366 100 366 0 0 67552 0 --:--:-- --:--:-- --:--:-- 73200 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> ``` ## 5. reference 1. [官方安裝資料](https://krew.sigs.k8s.io/docs/user-guide/setup/install/ "install") 2. [Kubectl plugins available](https://krew.sigs.k8s.io/plugins/ "Kubectl plugins available") 3. [中國網友資料](https://cloud.tencent.com/developer/article/1802230 "中國網友資料") 4. [Security Context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/ "Configure a Security Context for a Pod or Container") 5. [cyclonus](https://github.com/mattfenwick/kubectl-cyclonus "cyclonus") 6. [museum-dino-kiosk sample參考資料](https://hackmd.io/YqyhuPAvTymwHhu2lWp22Q?view)