# containerd failed to prepare extraction snapshot ## 報錯 * 手動拉 image 時發現以下報錯 ``` $ crictl pull 192.168.11.90/suse-observability/hadoop:3.4.1-java11-8-90a9d727 E0317 14:33:26.388176 8105 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"192.168.11.90/suse-observability/hadoop:3.4.1-java11-8-90a9d727\": failed to prepare extraction snapshot \"extract-289854180-amZ- sha256:0bd96a6a52b13edad37e66c3e91231992ee46a4801ffde00f47058b14fbea34e\": failed to stat parent: stat /var/lib/rancher/rke2/agent/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/121/fs: no such file or directory" image="192.168.11.90/suse-observability/hadoop:3.4.1-java11-8-90a9d727" FATA[0000] pulling image: failed to pull and unpack image "192.168.11.90/suse-observability/hadoop:3.4.1-java11-8-90a9d727": failed to prepare extraction snapshot "extract-289854180-amZ- sha256:0bd96a6a52b13edad37e66c3e91231992ee46a4801ffde00f47058b14fbea34e": failed to stat parent: stat /var/lib/rancher/rke2/agent/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/121/fs: no such file or directory ``` ![image](https://hackmd.io/_uploads/Sk788BB21e.png) * 查看 containerd log ![image](https://hackmd.io/_uploads/B1lw8rS3kl.png) ## 解決 * 檢查 snapshot 有發現 extract 代表這個 snapshot 有異常 ``` $ ctr --address /run/k3s/containerd/containerd.sock -n k8s.io snapshots ls | grep extract WARN[0000] DEPRECATION: The `configs` property of `[plugins."io.containerd.grpc.v1.cri".registry]` is deprecated since containerd v1.5 and will be removed in containerd v2.1. Use `config_path` instead. extract-274496232-mJ-Z sha256:0507e189b1b568447fd80c777b80b2f7911f4c3082ab599d836a51bab3573e3d sha256:3d9824a6b05ece6f0b450dbca8416cece5d4906739d71a8c93972382c4cad249 Active extract-723398845-17rV sha256:0bd96a6a52b13edad37e66c3e91231992ee46a4801ffde00f47058b14fbea34e sha256:1eef8b11f2f42810e7909b4de7b3d8bcf6232598d4e052f2b7094ab67ed8257f Active ``` * 停止 rke2-agent or rke2-server 服務，刪除 `/var/lib/rancher/rke2/agent/containerd` 目錄讓他重新生成 ``` $ systemctl stop rke2-agent.service $ rm -rf /var/lib/rancher/rke2/agent/containerd $ systemctl start rke2-agent.service ``` * 異常的 snapshot 已經消失 ``` $ ctr --address /run/k3s/containerd/containerd.sock -n k8s.io snapshots ls | grep extract WARN[0000] DEPRECATION: The `configs` property of `[plugins."io.containerd.grpc.v1.cri".registry]` is deprecated since containerd v1.5 and will be removed in containerd v2.1. Use `config_path` instead. ``` * 並且可以正常拉到 image ``` $ crictl pull 192.168.11.90/suse-observability/hadoop:3.4.1-java11-8-90a9d727 Image is up to date for sha256:abd67bc1122167df5881d6f8ecfc27e711a04d49743edbb3e91916f69960df3d ``` ## 參考 https://serverfault.com/questions/1166840/k3s-failed-to-prepare-extraction-snapshot