Kubernetes Removals and Deprecations In 1.24

# Kubernetes Removals and Deprecations In 1.24 Thursday, March 31, 2022 **Authors**: Mickey Boxell (Oracle), Debabrata Panigrahi (Independent) As Kubernetes evolves, features and APIs are periodically revisited and removed. New features may offer an alternative or improved approach to solving an existing problem, motivating the team to remove the old method. Rather than immediately being removed, old APIs are first deprecated and then removed according to the [Kubernetes Deprecation Policy](https://kubernetes.io/docs/reference/using-api/deprecation-policy/). We want to make sure you are aware of the changes coming in the Kubernetes 1.24 release. Kubernetes 1.24 is scheduled to be released in April and includes several removed and deprecated APIs. **Removed** APIs are those that you can use in current, supported Kubernetes versions but have already been deprecated. These removals have been superseded by newer, stable/generally available (GA) APIs. **Deprecated** APIs have been marked for removal in a future Kubernetes release. For an exhaustive list of the changes coming in Kubernetes 1.24, take a look at the [CHANGELOG](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md). ## API removals and deprecations for Kubernetes 1.24 ### Removals The 1.24 release will stop serving the API versions listed below. * [Dynamic Kubelet Configuration](https://github.com/kubernetes/enhancements/issues/281):`DynamicKubeletConfig` is used to enable the dynamic configuration of kubelet. The `DynamicKubeletConfig` flag was deprecated in Kubernetes 1.22. As of 1.24, the feature gate was removed from kubelet. See [Reconfigure kubelet](/docs/tasks/administer-cluster/reconfigure-kubelet/). Refer to the ["Dynamic kubelet config is removed" KEP](https://github.com/kubernetes/enhancements/issues/281) for more information. * [Removing dockershim from kubelet](https://github.com/kubernetes/enhancements/issues/2221) (more on this below): the Container Runtime Interface (CRI) for Docker (i.e., Dockershim) is currently a built-in container runtime in the kubelet code base. It was deprecated in 1.20. As of 1.24, dockershim has been removed from kubelet. Check out this blog on [what you need to do to have a working cluster for 1.24](https://github.com/kubernetes/website/issues/30917). * [VolumeSnapshot v1beta1 CRD will be removed in 1.24](https://github.com/kubernetes/enhancements/issues/177). Volume snapshot and restore functionality for Kubernetes and the [Container Storage Interface](https://github.com/container-storage-interface/spec/blob/master/spec.md) (CSI), which provides standardized APIs design (CRDs) and adds PV snapshot/restore support for CSI volume drivers, entered beta in 1.20. VolumeSnapshot v1beta1 was deprecated in 1.21. Refer to [KEP-177: CSI Snapshot](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/177-volume-snapshot#kep-177-csi-snapshot) and [kubernetes-csi/external-snapshotter](https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v4.1.0) for more information. ### Deprecations The 1.24 release deprecates the API versions listed below. * [Kubernetes system components logs sanitization](https://github.com/kubernetes/enhancements/issues/1753): This feature introduced a logging filter that could be applied to all Kubernetes system components logs to prevent various types of sensitive information from leaking via logs. An alternative approach to ensuring secrets are not being logged, one which implemented static analysis as part of a presubmit CI graduated to stable in 1.23. Dynamic log sanitization is deprecated as of 1.24. Refer to [KEP-1753: Kubernetes system components logs sanitization](https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/1753-logs-sanitization#deprecation) for more information. * In-tree provisioner to CSI driver migration. This applies to a number of in-tree plugins, including [vSphere](https://github.com/kubernetes/enhancements/issues/1491) and [Azure File](https://github.com/kubernetes-sigs/azurefile-csi-driver). Refer to the [In-tree Storage Plugin to CSI Migration Design Doc](https://github.com/kubernetes/design-proposals-archive/blob/main/storage/csi-migration.md#background-and-motivations) for more information. * [Storage capacity tracking for pod scheduling](https://github.com/kubernetes/enhancements/issues/1472): The CSIStorageCapacity API supports exposing currently available storage capacity via CSIStorageCapacity objects and enhances scheduling of pods that use CSI volumes with late binding. The CSIStorageCapacity API is stable as of 1.24. The v1beta1 CSIStorageCapacity API is deprecated in 1.24. Refer to the [Storage Capacity Constraints for Pod Scheduling KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1472-storage-capacity-tracking) for more information. ### A note about Dockershim It's safe to say that the removal receiving the most attention with the release of Kubernetes 1.24 is Dockershim. Dockershim was deprecated in 1.20. As noted in the [Kubernetes 1.20 changelog](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md#deprecation): "Docker support in the kubelet is now deprecated and will be removed in a future release. The kubelet uses a module called "dockershim" which implements CRI support for Docker and it has seen maintenance issues in the Kubernetes community." With the upcoming release of Kubernetes 1.24, that day is finally here. In the article, [Don't Panic: Kubernetes and Docker](https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/), the authors succinctly captured the change's impact and encouraged users to remain calm: "Docker as an underlying runtime is being deprecated in favor of runtimes that use the Container Runtime Interface (CRI) created for Kubernetes. Docker-produced images will continue to work in your cluster with all runtimes, as they always have." Several guides have been created with helpful information about migrating from dockershim to other container runtimes. You can find them on the [Migrating from dockershim](https://kubernetes.io/docs/tasks/administer-cluster/migrating-from-dockershim/) page in the Kubernetes documentation. For more information about why Kubernetes is moving away from dockershim, check out the aptly named: [Kubernetes is Moving on From Dockershim](https://kubernetes.io/blog/2022/01/07/kubernetes-is-moving-on-from-dockershim/) and the [updated dockershim removal FAQ](https://kubernetes.io/blog/2022/02/17/dockershim-faq/). Take a look at the [What you need to do to have a working cluster for 1.24](https://github.com/kubernetes/website/issues/30917) to learn what you need to do to go from a working 1.23 cluster to a working 1.24 cluster. ## What to do ### `kubectl convert` As mentioned in [prior API removal posts](https://kubernetes.io/blog/2021/07/14/upcoming-changes-in-kubernetes-1-22/#kubectl-convert), [`kubectl-convert`](https://kubernetes.io/docs/tasks/tools/included/kubectl-convert-overview/), a plugin for the `kubectl` Kubernetes command-line tool, can be helpful to address migrating off deprecated APIs. It does so by facilitating the conversion of manifests between different API versions, for example, from a deprecated to a non-deprecated API version. More general information about the API migration process can be found in the [Deprecated API Migration Guide](https://kubernetes.io/docs/reference/using-api/deprecation-guide/). ## The Kubernetes API removal process Kubernetes contains a large number of components that evolve over time. In some cases, this evolution results in APIs, flags, or entire features, being removed. To prevent users from facing breaking changes, Kubernetes adopted a feature [deprecation policy](https://kubernetes.io/docs/reference/using-api/deprecation-policy/). This policy ensures that stable APIs may only be deprecated when a newer stable version of that same API is available and that APIs have a minimum lifetime depending on their stability level: * Generally available (GA) or stable API versions may be marked as deprecated but must not be removed within a major version of Kubernetes. * Beta or pre-release API versions must be supported for 3 releases after deprecation. * Alpha or experimental API versions may be removed in any release without prior deprecation notice. Kubernetes will continue to remove APIs by following the deprecation policy regardless of whether an API is removed due to a beta feature graduating to stable or because that API was not proven to be successful and will also make sure migration options are documented. ### Looking ahead The Kubernetes 1.25 and 1.26 releases planned for later this year will stop serving beta versions of several currently stable Kubernetes APIs. The 1.25 release will also remove PodSecurityPolicy, which was deprecated with Kubernetes 1.21 and will not graduate to stable. See [PodSecurityPolicy Deprecation: Past, Present, and Future](https://kubernetes.io/blog/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/) for more information. The official [list of API removals planned for Kubernetes 1.25](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-25) is: * The beta CronJob API (batch/v1beta1) * The beta EndpointSlice API (discovery.k8s.io/v1beta1) * The beta Event API (events.k8s.io/v1beta1) * The beta HorizontalPodAutoscaler API (autoscaling/v2beta1) * The beta PodDisruptionBudget API (policy/v1beta1) * The beta PodSecurityPolicy API (policy/v1beta1) * The beta RuntimeClass API (node.k8s.io/v1beta1) The official [list of API removals planned for Kubernetes 1.26](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-26) is: * The beta FlowSchema and PriorityLevelConfiguration APIs (flowcontrol.apiserver.k8s.io/v1beta1) * The beta HorizontalPodAutoscaler API (autoscaling/v2beta2) ## Want to know more? Deprecations are announced in the Kubernetes release notes. You can see the announcements of pending deprecations in the release notes for: * [Kubernetes 1.21](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md#deprecation) * [Kubernetes 1.22](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.22.md#deprecation) * [Kubernetes 1.23](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#deprecation) * [Kubernetes 1.24](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#deprecation) For information on the process of deprecation and removal, check out the official Kubernetes [deprecation policy](/docs/reference/using-api/deprecation-policy/#deprecating-parts-of-the-api) document.