# Upgrading OLM on k8s

* using `operator-sdk olm install` requires removing the pre-flight check of an existing OLM installed.
* even with that removed, it still fails
* Using upstream `install.sh` script, also requires removing pre-flight check.
* Have to remove the pre-flight check
* changed `kubectl create` to `kubectl apply`

### Steps
1. `kind create cluster`
2. `operator-sdk olm install --version v0.21.2`
3. `kubectl apply -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/crds.yaml`
4. `kubectl wait --for=condition=Established -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/crds.yaml`
5. `kubectl apply -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/olm.yaml`
6. `operator-sdk olm status`
7. `kubectl rollout status -w deployment/olm-operator --namespace=olm`
8. `kubectl rollout status -w deployment/catalog-operator --namespace=olm`
9. `kubectl rollout status -w deployment/packageserver --namespace=olm`

### Results
So if I run `operator-sdk olm install --version v0.21.2` then run `./install.sh v0.22.0` I get the following errors:

[jesusr@transam operator-sdk{master}]$ ./install.sh v0.22.0                                       
Warning: resource customresourcedefinitions/catalogsources.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should
 only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com configured
Warning: resource customresourcedefinitions/clusterserviceversions.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl appl
y should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
Warning: resource customresourcedefinitions/installplans.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should o
nly be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com configured
Warning: resource customresourcedefinitions/olmconfigs.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should onl
y be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com configured
Warning: resource customresourcedefinitions/operatorconditions.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply sh
ould only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com configured
Warning: resource customresourcedefinitions/operatorgroups.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should
 only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com configured
Warning: resource customresourcedefinitions/operators.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only
 be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com configured
Warning: resource customresourcedefinitions/subscriptions.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should 
only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com configured
The CustomResourceDefinition "clusterserviceversions.operators.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

v0.21.2 still shows as installed.

$ operator-sdk olm status                                    
INFO[0000] Fetching CRDs for version "v0.21.2"                                                    
INFO[0000] Fetching resources for resolved version "v0.21.2" 
INFO[0000] Successfully got OLM status for version "v0.21.2" 

Running the install script a SECOND time, seems to look better:

$ ./install.sh v0.22.0
customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com unchanged
Warning: resource customresourcedefinitions/clusterserviceversions.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com unchanged
The CustomResourceDefinition "clusterserviceversions.operators.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

But SDK still shows v0.21.2 installed

### Running kubectl from script by hand results

$ kubectl apply -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/crds.yaml
Warning: resource customresourcedefinitions/catalogsources.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com configured
Warning: resource customresourcedefinitions/clusterserviceversions.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
Warning: resource customresourcedefinitions/installplans.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com configured
Warning: resource customresourcedefinitions/olmconfigs.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com configured
Warning: resource customresourcedefinitions/operatorconditions.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com configured
Warning: resource customresourcedefinitions/operatorgroups.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com configured
Warning: resource customresourcedefinitions/operators.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com configured
Warning: resource customresourcedefinitions/subscriptions.operators.coreos.com is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com configured
The CustomResourceDefinition "clusterserviceversions.operators.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

#### wait

`kubectl wait --for=condition=Established -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/crds.yaml`

customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/clusterserviceversions.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com condition met

#### install olm yaml
`kubectl apply -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/olm.yaml`

Warning: resource namespaces/olm is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
Warning: existing pods in namespace "olm" violate the new PodSecurity enforce level "restricted:latest"
Warning: catalog-operator-86d5666fc7-55tql (and 4 other pods): allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
namespace/olm configured
Warning: resource namespaces/operators is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
namespace/operators configured
Warning: resource serviceaccounts/olm-operator-serviceaccount is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
serviceaccount/olm-operator-serviceaccount configured
Warning: resource clusterroles/system:controller:operator-lifecycle-manager is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrole.rbac.authorization.k8s.io/system:controller:operator-lifecycle-manager configured
Warning: resource clusterrolebindings/olm-operator-binding-olm is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrolebinding.rbac.authorization.k8s.io/olm-operator-binding-olm configured
Warning: resource olmconfigs/cluster is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
olmconfig.operators.coreos.com/cluster configured
Warning: resource deployments/olm-operator is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/olm-operator configured
Warning: resource deployments/catalog-operator is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/catalog-operator configured
Warning: resource clusterroles/aggregate-olm-edit is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrole.rbac.authorization.k8s.io/aggregate-olm-edit configured
Warning: resource clusterroles/aggregate-olm-view is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrole.rbac.authorization.k8s.io/aggregate-olm-view configured
Warning: resource operatorgroups/global-operators is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
operatorgroup.operators.coreos.com/global-operators configured
Warning: resource operatorgroups/olm-operators is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
operatorgroup.operators.coreos.com/olm-operators configured
Warning: resource clusterserviceversions/packageserver is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterserviceversion.operators.coreos.com/packageserver configured
Warning: resource catalogsources/operatorhubio-catalog is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
catalogsource.operators.coreos.com/operatorhubio-catalog configured

#### Seems to work by hand

operator-sdk olm status
INFO[0000] Fetching CRDs for version "v0.22.0"          
INFO[0000] Fetching resources for resolved version "v0.22.0" 
INFO[0000] Successfully got OLM status for version "v0.22.0"