# **Airshipctl Nodes rolling upgrade**

We will take an example of upgrading containerd.

**Verify current version of containerd on any node**


**Generate new qcow_bundle image with upgraded containerd version**

1. In image builder project, update containerd latest version in file image-builder/assets/playbooks/roles/multistrap/defaults/main.yaml


And get your PS merged. It will generate new qcow-bundle docker image.
Get the latest qcow_bundle docker image url from airship-image-builder-publish-latest job.


2. Update qcow_bundle Image url in airshipctl/manifests/function/airshipctl-base-catalogues/versions-airshipctl.yaml


**Update qcow_bundle image in ironic**

1. Export kubeconfig
export KUBECONFIG=/home/pallav/.airship/kubeconfig

2. Change current context

3. Delete ironic deployment

4. Verify Ironic deployment deleted

5. Run initinfra-target phase to deploy ironic with latest images

pallav@ubuntu:~/airshipctl$ airshipctl phase run initinfra-target

6. Verify Ironic is up and running after upgrade

**Upgrade control plane nodes excluding first node**

Below given steps for deprovisioning and reprovisioning for each control plane node one by one:


**Note:** Below patch is a workaround for bug in capm3 and it’s resolved in capm3 v0.4.2. So if you have capm3 version >= v0.4.2 then this workaround is not needed.

kubectl patch bmh <node-name> -n target-infra --type merge -p '{"spec": {"networkData": {"name": <node-name>-network-data", "namespace": "target-infra"}}}'

After deprovisioning finishes, Control plane node will again start provisioning which will have new qcow image as host image.



**Verify Containerd version in reprovisioned node**


**Upgrade First control plane node**

Requirements:

* API Server VIP

Steps:

1. Manifest Changes, Moving Ironic out of First target node

Change provisioning ip from First CP node IP to Second CP node IP

~/airshipctl/manifests/site/test-site/target/catalogues/networking.yaml
~/airshipctl/manifests/site/test-site/target/initinfra/patch_bmo_config.yaml
~/airshipctl/manifests/site/test-site/target/catalogues/versions-airshipctl.yaml
~/airshipctl/manifests/site/test-site/target/controlplane/metal3machinetemplate.yaml
~/airshipctl/manifests/function/k8scontrol/cluster.yaml
~/airshipctl/manifests/function/airshipctl-base-catalogues/networking.yaml

2. Add node01 in host-generation, so that node01 (first target) will be deployed as controlplane node using rolling upgrade.

~/airshipctl/manifests/site/test-site/target/controlplane/hostgenerator/host-generation.yaml

3. Edit configuration

kubectl edit cm ironic-vars -n metal3
kubectl edit metal3machinetemplate -n target-infra

4. Delete First Target Node

kubectl delete machine cluster-controlplane-lsvx5 -n target-infra

5. Run controlplane-target phase

airshipctl phase run controlplane-target

Here we have an issue,

Ironic is able to pxe boot and inspection completed successfully and machine is in ready state. After that:

Machine turned down and stuck.
Provisioning Stopped.
No Error observed.

We can try few things:

1. Try above procedure with new release of capm3 v1alpha5 and capi v1alpha4.
https://review.opendev.org/c/airship/airshipctl/+/804834

I've tried but ironic failed to deployed



2. Use ironic Provision IP as VIP.