---
layout: blog
title: 'Kubernetes Removals, Deprecations, and Major Changes in Kubernetes 1.29'
date: 2023-11-09
slug: kubernetes-1-29-release-announcement
---

**Authors:** Carol Valencia, Kristin Martin, Abigail McCarthy, James Quigley, Hosam Kamel

# Kubernetes Removals, Deprecations, and Major Changes in Kubernetes 1.29

As with every release, Kubernetes v1.29 will introduce some feature deprecations and removals. Our continued ability to produce high-quality releases is a testament to our robust development cycle and healthy community. The following list are of some of the deprecations and removals coming in the Kubernetes 1.29 release.

## The Kubernetes API Removal and Deprecation process

The Kubernetes project has a well-documented deprecation policy for features. This policy states that stable APIs may only be deprecated when a newer, stable version of that same API is available and that APIs have a minimum lifetime for each stability level. A deprecated API is one that has been marked for removal in a future Kubernetes release; it will continue to function until removal (at least one year from the deprecation), but usage will result in a warning being displayed. Removed APIs are no longer available in the current version, at which point you must migrate to using the replacement.

* Generally available (GA) or stable API versions may be marked as deprecated, but must not be removed within a major version of Kubernetes.
* Beta or pre-release API versions must be supported for 3 releases after deprecation.
* Alpha or experimental API versions may be removed in any release without prior deprecation notice.

Whether an API is removed as a result of a feature graduating from beta to stable or because that API simply did not succeed, all removals comply with this deprecation policy. Whenever an API is removed, migration options are communicated in the documentation.

## A note about the k8s.gcr.io redirect to registry.k8s.io

To host its container images, the Kubernetes project uses a community-owned image registry called registry.k8s.io. Starting last March traffic to the old k8s.gcr.io registry began being redirected to registry.k8s.io. The deprecated k8s.gcr.io registry will eventually be phased out. For more details on this change or to see if you are impacted, please read [k8s.gcr.io Redirect to registry.k8s.io - What You Need to Know](https://kubernetes.io/blog/2023/03/10/image-registry-redirect/).

## Deprecations and Removals for Kubernetes v1.29

### Removing In-Tree Cloud Providers [KEP-2395](https://github.com/kubernetes/enhancements/issues/2395)

The [feature gates](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) `DisableCloudProviders` and `DisableKubeletCloudCredentialProviders` are now set to `true` by default. This change will require that users who are currently using in-tree cloud providers (Azure, GCE, or vSphere) to enable external cloud controller managers, or to set the feature gates to `false`. Enabling external cloud controller managers requires setting the command line flag `--cloud-provider=external` on the kubelet, kube-apiserver, and kube-controller-manager in addition to running the cloud controller manager.

For more information about how to enable and run external cloud controller managers, please see the Kubernetes documentation [Cloud Controller Manager Administration](https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/) and [Migrate Replicated Control Plane To Use Cloud Controller Manager](https://kubernetes.io/docs/tasks/administer-cluster/controller-manager-leader-migration/).

For general information about cloud controller managers, please see the Kubernetes document
[Cloud Controller Manager](https://kubernetes.io/docs/concepts/architecture/cloud-controller/).

### Removal of `v1beta2` flow control API group

The flowcontrol.apiserver.k8s.io/v1beta2 API version of FlowSchema and PriorityLevelConfiguration will no longer be [served in v1.29](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-29).

To prepare for this, you can edit your existing manifests and rewrite client software to use the `flowcontrol.apiserver.k8s.io/v1beta3` API version, available since v1.26. All existing persisted objects are accessible via the new API. Notable changes in `flowcontrol.apiserver.k8s.io/v1beta3` include
the PriorityLevelConfiguration spec.limited.assuredConcurrencyShares field is renamed to spec.limited.nominalConcurrencyShares.

### Deprecation of `status.nodeInfo.kubeProxyVersion` field of v1.Node

The `.status.kubeProxyVersion` field for Node objects will be [marked as deprecated](https://github.com/kubernetes/enhancements/issues/4004) in v1.29 in preparation for its removal in a future release. This field is not accurate and is set by kubelet, which does not actually know the kube-proxy version, or even if kube-proxy is running.

### Deprecation of the `SkipReadOnlyValidationGCE` feature gate

The `SkipReadOnlyValidationGCE` feature gate will be [deprecated](https://github.com/kubernetes/kubernetes/pull/121083). Previous versions of Kubernetes on Google Cloud required that workloads (for example Deployments, DaemonSets, etc.) which used PersistentDisk volumes were using them in read-only mode. This validation provided very little value at a relatively high host implementation cost, and will no longer be validated. If this is a problem for a specific use-case, you'll be able to the set SkipReadOnlyValidationGCE gate to false so as to re-enable the validation, and then we hope you would file a Kubernetes bug with details.

## Major Changes for Kubernetes v1.29

### Native sidecar containers promoted to beta

Introduced in alpha in Kubernetes v1.28, native sidecare containers will be [beta](https://github.com/kubernetes/enhancements/issues/753) in Kubernetes v1.29. In addition to fixing known issues, improvements include implementing proper termination ordering and allowing sidecar containers to restart during the shutdown of the Pod.

## Looking Ahead

The official list of [API removals](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-29) planned for Kubernetes v1.29 includes:

* The flowcontrol.apiserver.k8s.io/v1beta2 API version of FlowSchema and PriorityLevelConfiguration will no longer be served in v1.29.

### Want to know more?

Deprecations are announced in the Kubernetes release notes. You can see the announcements of pending deprecations in the release notes for:

* [Kubernetes v1.25](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#deprecation)
* [Kubernetes v1.26](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.26.md#deprecation)
* [Kubernetes v1.27](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.27.md#deprecation)
* [Kubernetes v1.28](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.28.md#deprecation)

We will formally announce the deprecations that come with [Kubernetes v1.29](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.29.md#deprecation) as part of the CHANGELOG for that release.

For information on the deprecation and removal process, check out the official Kubernetes [deprecation policy](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecating-parts-of-the-api) document.