Cluster API Scale From Zero Provider Status as of September 2024

Capturing some of the work that will need to be done to expand the coverage of the opt-in autoscaling from zero feature in Cluster API.

Subsections of this document will cover specific providers and the necessary changes in each to support scale from zero as an automated process. The overall process for adding capacity information is as follows:

• reconcile infrastructure machine template resources for provider
• look up instance information from infrastructure machine template
• add status.capacity to the infrastructure machine template

Open Questions

• For any given provider, what is the most convenient/efficient way to learn the capacity information?
  • on some providers this may involve a call to a cloud service

Providers

The providers listed here have been sourced from those available in the kubernetes-sigs organization on github.

Kubemark

The status updates have been implemented in this provider and could be used as an example for other providers.

Docker

https://github.com/kubernetes-sigs/cluster-api/tree/main/test/infrastructure/docker

Although the docker provider is primarily used for testing, adding capacity functionality will help with testing the overall feature.

• add status.capacity to DockerMachineTemplate API type
• add a reconciler for DockerMachineTemplates
• add capacity information to status on first reconcile

• capacity information needs a source
  • need a way to specify the capacity limits of a docker machine
  • is this respected by docker? (don't think so)

AWS

https://github.com/kubernetes-sigs/cluster-api-provider-aws

• add status.capacity to AWSMachineTemplate API type
• add a reconciler for AWSMachineTemplates
• add capacity information to status on first reconcile

• capacity information should come from metadata service

Azure

https://github.com/kubernetes-sigs/cluster-api-provider-azure

• add status.capacity to AzureMachineTemplate API type
• add/supplement reconciler for AzureMachineTemplates
  • there is a controller for AzureJsonTemplateReconciler which appears to reconcile the AzureMachineTemplates. unsure if this would be an appropriate place to gather instance information.
  • talked with @mboersma in #cluster-api-azure slack channel, it could work but adding a new controller might also be cleaner.
• add capacity information to status on first reconcile

• capacity information should come from metadata service

CloudStack

https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack

• add status.capacity to CloudStackMachineTemplate API type
• add a reconciler for CloudStackMachineTemplates
• add capacity information to status on first reconcile

• capacity information should come from metadata service

Digital Ocean

https://github.com/kubernetes-sigs/cluster-api-provider-digitalocean

• add status.capacity to DOMachineTemplate API type
• add a reconciler for DOMachineTemplates
• add capacity information to status on first reconcile

• capacity information should come from metadata service

GCP

https://github.com/kubernetes-sigs/cluster-api-provider-gcp

• add status.capacity to GCPMachineTemplate API type
• add a reconciler for GCPMachineTemplates
• add capacity information to status on first reconcile

• capacity information should come from metadata service

IBMCloud

https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud

implemented

good example for other providers

KubeVirt

https://github.com/kubernetes-sigs/cluster-api-provider-kubevirt

• add status.capacity to KubevirtMachineTemplate API type
• add a reconciler for KubevirtMachineTemplates
• add capacity information to status on first reconcile

• not sure where capacity information comes from, need to research kubevirt more

OpenStack

https://github.com/kubernetes-sigs/cluster-api-provider-openstack

• add status.capacity to OpenStackMachineTemplate API type
• add a reconciler for OpenStackMachineTemplates
• add capacity information to status on first reconcile

• capacity information should come from nova service(?)

Packet (Equinix Metal)

https://github.com/kubernetes-sigs/cluster-api-provider-packet

• add status.capacity to PacketMachineTemplate API type
• add a reconciler for PacketMachineTemplates
• add capacity information to status on first reconcile

• capacity information should come from metadata service

vSphere

https://github.com/kubernetes-sigs/cluster-api-provider-vsphere

implemented

good example for other providers, slightly more complex than ibmcloud

• ask about vmware version at office hours