``` # Assumption1: core bootstrap packages can't be removed # Assumption2: core bootstrap packages can be identified with field name # Assumption3: ClusterBootstrapTemplate will roll out some additional packages like Pinniped, Metrics-Server in the first place # # Question1: How can we identify additional packages? There is no key to identify it so that we could uniquely link the pkg in older version to the same pkg in the newer version. # Question2: Do we want to add a field that indicates it's a unmanaged package that can be set by user ``` Before the upgrade merge happens ``` yaml apiVersion: run.tanzu.vmware.com/v1alpha3 kind: TanzuClusterBootstrap metadata: name: <cluster-name> namespace: <cluster-namespace> spec: # Case 1: users use their inline value/secretRef cni: refName: "cilium.tce.vmware.com.1.18.2+vmware.1-tkg.1-rc.1" valuesFrom: inline: datapathMode: veth bgp: enabled: true announce: loadbalancerIP: true podCIDR: true # Case 2: User doesn't do customization cpi: refName: "vsphere-cpi.tanzu.vmware.com.1.21.2+vmware.1-tkg.1" valuesFrom: providerRef: apiVersion: cpi.tanzu.vmware.com/v1alpha1 kind: VsphereCPIConfig name: v1.21.2---vmware.1-tkg.1 # Case 3: User changes the name of providerRef csi: refName: "vsphere-csi.tanzu.vmware.com.2.3.0+vmware.1-tkg.4-rc.1" valuesFrom: providerRef: <ObjectReference> apiVersion: csi.tanzu.vmware.com/v1alpha1 kind: VsphereCSIConfig name: someCustomName # Case 4: User changes the package they are using (packageRef) kapp: refName: "kapp-controller.user.custom.com.latest" valuesFrom: providerRef: <ObjectReference> apiVersion: csi.tanzu.vmware.com/v1alpha1 kind: KappConfig name: v1.21.2---vmware.1-tkg.1 additionalPackages: # Case 5: The packageRef is not changed by user, use packageRef as UID and handle it the same way as core packages (Pending with Question1) - refName: "pinniped.tanzu.vmware.com.0.12.0+vmware.1-tkg.1-rc.1" valuesFrom: #either from or inline secretRef: <secretName> #either this or ref providerRef: <ObjectReference> apiVersion: supervisorauth.cpbu.tanzu.vmware.com/v1alpha1 kind: PinnipedConfig name: vsphere-sso-config inline: # map[string]interface{}, validated with package schema infrastructure_provider: "vsphere" tkg_cluster_role: "workload" identity_management_type: "oidc" pinniped: cert_duration: "" cert_renew_before: "" supervisor_svc_endpoint: "" supervisor_ca_bundle_data: "" # Case 6: Package is removed in the next release (Pending with Question1) - refName: "metrics-server.tanzu.vmware.com.0.5.1+vmware.1-tkg.1-rc.1" # Case 7: PackageRef is changed by user (Pending with Question1) - refName: "secretgen-controller.tanzu.vmware.com.changed" ``` After merging with new version ``` yaml apiVersion: run.tanzu.vmware.com/v1alpha3 kind: TanzuClusterBootstrap metadata: name: <cluster-name> namespace: <cluster-namespace> spec: # Case 1: We don't update CNI at all cni: refName: "cilium.tce.vmware.com.1.18.2+vmware.1-tkg.1-rc.1" valuesFrom: inline: datapathMode: veth bgp: enabled: true announce: loadbalancerIP: true podCIDR: true # Case 2: We will update the CPI according to the new template cpi: refName: "vsphere-cpi.tanzu.vmware.com.1.22.3+vmware.1-tkg.2-rc.1" valuesFrom: providerRef: apiVersion: cpi.tanzu.vmware.com/v1alpha1 kind: VsphereCPIConfig name: v1.22.3+vmware.1-tkg.2-rc.1 # Case 3: We will update the packageRef according to the new template, while keep the providerRef name changed by user. However, if user changes apiVersion or Kind, we will skip updating this package csi: refName: "vsphere-csi.tanzu.vmware.com.2.4.0+vmware.1-tkg.44" valuesFrom: providerRef: <ObjectReference> apiVersion: csi.tanzu.vmware.com/v1alpha1 kind: VsphereCSIConfig name: someCustomName # Case 4: Skip updating this package kapp: refName: "kapp-controller.user.custom.com.latest" valuesFrom: providerRef: <ObjectReference> apiVersion: csi.tanzu.vmware.com/v1alpha1 kind: KappConfig name: v1.21.2---vmware.1-tkg.1 additionalPackages: # Case 5: The packageRef is not changed by user, use packageRef as UID and handle it the same way as core packages (Pending with Question1) - refName: "pinniped.tanzu.vmware.com.0.12.0+vmware.1-tkg.1-rc.1" valuesFrom: #either from or inline secretRef: <secretName> #either this or ref providerRef: <ObjectReference> apiVersion: supervisorauth.cpbu.tanzu.vmware.com/v1alpha1 kind: PinnipedConfig name: vsphere-sso-config inline: # map[string]interface{}, validated with package schema infrastructure_provider: "vsphere" tkg_cluster_role: "workload" identity_management_type: "oidc" pinniped: cert_duration: "" cert_renew_before: "" supervisor_svc_endpoint: "" supervisor_ca_bundle_data: "" # Case 6: Package removed # Case 7: Unchanged - refName: "secretgen-controller.tanzu.vmware.com.changed" ``` Webhook restriction assumptions: 1. For cni, cpi, csi, kapp once created a. we won’t allow packageRef’s to be downgraded or change the package from something like calico to antrea (How about using a user built package?) b. We can start with disallowing change of apiVersion and Kind. In the future we can relax this (Not allowing change from providerRef to secretRef/inline as well?) c. Can change inline or secret to whatever 2. For Additional packages that are created a. no deletion of a package allowed, in the future we can consider relaxing this b. Can bump package version b. Not allowed to change apiVersion and Kind for provider c. Can change inline or secret to whatever We still have the following valid cases: - case1 - case2