# Reasoning around deprecation/delegation in FBC upgrade graphs
Prior Art
[Deprecated Channels proposal](https://hackmd.io/xgPy6mErQ6WFoPVkLRdDfA?view) (v0, abandoned)
[PM's Deprecation Management](https://docs.google.com/document/d/1aDY93Vg-jnUvWqSeT5JXoV2272WgB0HaLTjMjmewhBo/)
## Deprecation
### arguments for a separate schema
1. comprehensibility, as a collection of standalone things it makes it easier to reason about them than sprinkling them amongst the graph of bundle+version relationship
2. preserves immutability of existing graphs; no author needs re-release their graph info to update, which can be important for;
3. [ref](https://redhat-internal.slack.com/archives/C3VS0LV41/p1697466208164929) deprecation announcements will likely be orthogonal to release schedules (internal ref to [current deprecation process](https://docs.engineering.redhat.com/display/CFC/Delivery#Delivery-deprecating-bundles), which is a removal process, not true deprecation)
### example schemas for deprecation
#### singleton schema
```yaml=
schema: olm.catalog.meta
deprecations:
- ref: foobar-v0.7
kind: bundle #scope indication#
message: |
v0.7 is deprecated. Uninstall and install new v0.9 for support.
- ref: foo
kind: package
message: |
package foo is end of life. Please use 'baz' package for support.
- ref: stable-v0 #some unique string#
kind: channel #scope indication#
message: |
channel stable-v0 is no longer supported. Please switch to channel 'stable-v1'.
```
#### multiple schemas
```yaml=
schema: olm.deprecation.package
ref: foo
message: |
package foo is end of life. Please use 'bar' package for support.
---
schema: olm.deprecation.channel
ref: stable-v0
message: |
channel stable-v0 is no longer supported. Please switch to channel 'stable-v1'.
---
schema: olm.deprecation.bundle
ref: foobar-v0.7
message: |
v0.7 is deprecated. Uninstall and install new v0.9 for support.
```
#### singleton, satisfying Meta schema
```yaml=
schema: olm.catalog.meta
properties:
- type: olm.deprecation
value:
scope: bundle
ref: foobar-v0.7
message: |
v0.7 is deprecated. Uninstall and install new v0.9 for support.
- type: olm.deprecation
value:
scope: package
ref: foo
message: |
package foo is end of life. Please use 'baz' package for support.
- type: olm.deprecation
value:
scope: channel
ref: stable-v0
message: |
channel stable-v0 is no longer supported. Please switch to channel 'stable-v1'.
```
#### multi, satisfying Meta schema
```yaml=
schema: olm.deprecation.bundle
properties:
- type: olm.deprecation.bundle.property
value:
ref: foobar-v0.7
message: |
v0.7 is deprecated. Uninstall and install new v0.9 for support.
---
schema: olm.deprecation.package
properties:
- type: olm.deprecation.package.property
value:
ref: foo
message: |
package foo is end of life. Please use 'baz' package for support.
---
schema: olm.deprecation.channel
properties:
- type: olm.deprecation
value:
ref: stable-v0
message: |
channel stable-v0 is no longer supported. Please switch to channel 'stable-v1'.
```
#### singleton, Meta-satisfying, focused-scope schema:
<h5 style="text-align: center;color: red">winnah</h5>
1. requires package to be valid
2. can be 0..1 for each package to be valid
3. name is optional, and not validated
```yaml=
schema: olm.catalog.deprecations
package: foobar
name: ###if desired###
deprecations:
- schema: olm.bundle
name: foobar-v0.7
message: |
foobar-v0.7 is deprecated. Uninstall and install foobar-v0.9 for support.
- schema: olm.package
message: |
package foobar is end of life. Please use 'baz' package for support.
- schema: olm.channel
name: stable-v0
message: |
channel stable-v0 is no longer supported. Please switch to channel 'stable-v1'.
```
### example illustration of `opm alpha render-graph` output, with deprecated package, channel, and bundle.
note: package/channel styling doesn't appear to be supported in hackmd but here's what it should look like:
```mermaid
graph LR
classDef deprecated fill:#f9f
%% package "rhacs-operator"
subgraph "rhacs-operator"
%% channel "stable"
subgraph rhacs-operator-stable["stable"]
rhacs-operator-stable-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]
rhacs-operator-stable-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]
rhacs-operator-stable-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- replace --> rhacs-operator-stable-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]
rhacs-operator-stable-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- "skipRange(>= 4.0.0 < 4.1.0)" --> rhacs-operator-stable-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]
rhacs-operator-stable-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-stable-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- replace --> rhacs-operator-stable-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-stable-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- skip --> rhacs-operator-stable-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-stable-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- "skipRange(>= 4.0.0 < 4.1.1)" --> rhacs-operator-stable-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-stable-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- "skipRange(>= 4.0.0 < 4.1.1)" --> rhacs-operator-stable-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-stable-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-stable-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]-- replace --> rhacs-operator-stable-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-stable-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- skip --> rhacs-operator-stable-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-stable-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- "skipRange(>= 4.0.0 < 4.1.2)" --> rhacs-operator-stable-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-stable-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- "skipRange(>= 4.0.0 < 4.1.2)" --> rhacs-operator-stable-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-stable-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]-- "skipRange(>= 4.0.0 < 4.1.2)" --> rhacs-operator-stable-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-stable-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]:::deprecated
rhacs-operator-stable-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- "skipRange(>= 4.0.0 < 4.2.0)" --> rhacs-operator-stable-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
rhacs-operator-stable-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- "skipRange(>= 4.0.0 < 4.2.0)" --> rhacs-operator-stable-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
rhacs-operator-stable-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]-- "skipRange(>= 4.0.0 < 4.2.0)" --> rhacs-operator-stable-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
rhacs-operator-stable-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]-- "skipRange(>= 4.0.0 < 4.2.0)" --> rhacs-operator-stable-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
rhacs-operator-stable-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]-- replace --> rhacs-operator-stable-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
end
%% channel "candidate"
subgraph rhacs-operator-candidate["candidate"]
rhacs-operator-candidate-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]
rhacs-operator-candidate-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]
rhacs-operator-candidate-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- replace --> rhacs-operator-candidate-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]
rhacs-operator-candidate-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- "skipRange(>= 4.0.0 < 4.1.0)" --> rhacs-operator-candidate-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]
rhacs-operator-candidate-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-candidate-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- replace --> rhacs-operator-candidate-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-candidate-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- skip --> rhacs-operator-candidate-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-candidate-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- "skipRange(>= 4.0.0 < 4.1.1)" --> rhacs-operator-candidate-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-candidate-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- "skipRange(>= 4.0.0 < 4.1.1)" --> rhacs-operator-candidate-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]
rhacs-operator-candidate-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-candidate-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]-- replace --> rhacs-operator-candidate-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-candidate-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- skip --> rhacs-operator-candidate-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-candidate-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- "skipRange(>= 4.0.0 < 4.1.2)" --> rhacs-operator-candidate-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-candidate-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- "skipRange(>= 4.0.0 < 4.1.2)" --> rhacs-operator-candidate-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-candidate-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]-- "skipRange(>= 4.0.0 < 4.1.2)" --> rhacs-operator-candidate-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]
rhacs-operator-candidate-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]:::deprecated
rhacs-operator-candidate-rhacs-operator.v4.0.0["rhacs-operator.v4.0.0"]-- "skipRange(>= 4.0.0 < 4.2.0)" --> rhacs-operator-candidate-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
rhacs-operator-candidate-rhacs-operator.v4.1.0["rhacs-operator.v4.1.0"]-- "skipRange(>= 4.0.0 < 4.2.0)" --> rhacs-operator-candidate-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
rhacs-operator-candidate-rhacs-operator.v4.1.1["rhacs-operator.v4.1.1"]-- "skipRange(>= 4.0.0 < 4.2.0)" --> rhacs-operator-candidate-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
rhacs-operator-candidate-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]-- "skipRange(>= 4.0.0 < 4.2.0)" --> rhacs-operator-candidate-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
rhacs-operator-candidate-rhacs-operator.v4.1.2["rhacs-operator.v4.1.2"]-- replace --> rhacs-operator-candidate-rhacs-operator.v4.2.0["rhacs-operator.v4.2.0"]
end
end
style rhacs-operator-candidate fill:#c9c
style rhacs-operator fill:#a9a
```
### Open Questions about cross-catalog interactions (may need to be resolved in RFCs):
- What happens if two catalogs (foo, bar) disagree on whether a package is deprecated? OLMv0 provides spec.source and spec.sourceNamespace to disambiguate, but how do we leverage those and what are comparable v1 mechanisms?
- package uniqueness is enforced in opm validate so we could assert that package+channel+version tuple is sufficient to uniquely identify a candidate
- not enforced in opm render so we’re not consistent
- we say we want our v1 experience to be analogous to dnf/apt, where add’l contiguous versions could be made available by adding a new source, but then we say that catalogs cannot aggregate, which prevents it
#### A: deprecation is to be considered only w/in the context of the current catalog
- Do we error if someone puts delegation across catalogs?
#### A: delegation has been indefinitely deferred
installation of a thing which has deprecated dependencies -- should generate a warning, not ignore it
bundle authors are not required to create/publish a new bundle in order to deprecate -- bundle immutability
argument that v1 should allow for any-scoped delegation -- inclusive-first, with filtering out as missing specified
## Old delegation content
The desired experiences include:
authors can communicate sunset strategies to cluster admins with more gradations than moving from “in catalog” to “not in catalog and skipRange required to avoid stranding”
organization can maintain continuous support across disparate branch/versioning strategies
organization can perform quick-response releases which subordinate predecessor releases without having to pretend to be that release and without the risk of stranding some installations (this is also a case served by introducing a ‘release’ field in the bundle version, similar to RPM)
with expansion of delegates to include conditional expressions (for e.g. CEL to validate against a spec field) organization can test new objects in controlled, scaled deployments (i.e. dogfood, canary)
We want to represent these layered relationships in a way that doesn’t clutter catalog specs, and allows them to evolve independently (since the only data dependency on FBC schema is that they have string identifiers).
The ‘target’ end of these relationships will be expressed, so that there is never a need to retroactively update some object to flag it for delegation/deprecation itself. There is a security concern here in that we should have some way to verify the catalog contributor is an authorized originator of these relationships.
The relationships between the catalog and its meta-relationships would be collapsed in any opm render-like operation, with the resulting schema … [NB: consider this deeply, as replaces/skips mechanics could make this complex]
Summarize the RFC(s). Refine, clarify, reiterate the “need” and “benefit”.
Solution Design and Architecture
Metadata is represented as a JSON or YAML blob, either discrete from or co-resident with existing catalog data (like “undecorated” FBC, where no duplicate nodes are allowed). This will function as FBC does, where the data is composed of the whole hierarchical fs.FS.
```yaml=
schema: olm.catalog.meta
delegates:
- kind: bundle
ref: foo-stable-v1.0
delegate: foobar-v0.8
```
delegates a specific bundle+version (foo-stable-v1.0) with another (foobar-v0.8) by name
```yaml=
schema: olm.catalog.meta
delegates:
- kind: channel
ref: stable-v0
delegate: foobar-fast
```
delegates a specific channel (stable-v0) with another (foobar-fast) by name
```yaml=
schema: olm.catalog.meta
delegates:
- id: stable-v1
kind: channelCondition
condition: (insert CEL here)
delegate: foobar-canary
```
delegates a channel to be replaced by another (foobar-canary) if a condition evaluates true
```yaml=
schema: olm.catalog.meta
deprecations:
- id: foobar-v0.8
kind: bundleRange
bundleRange: <0.6
```
deprecates a range of bundle versions
```yaml=
schema: olm.catalog.meta
deprecations:
- id: foobar-v0.7
kind: bundle
bundle: foobar-v0.7
```
deprecates a specific bundle version
```yaml=
schema: olm.catalog.meta
deprecations:
- id: baz
package: foo
```
deprecates a package by name
-----------------
olm.catalog.meta constraints:
(unambiguous intent) a delegate shall not delegate for both a bundle and a channel
(ordinality) a delegate shall only delegate for a single instance of the selected type
(brevity) a deprecated bundle entry can indicate a single bundle version or a valid semver range expression
(target uniqueness) a deprecated bundle entry shall only indicate one olm.bundle version (e.g. no regexes, globs)
(edge uniqueness) there shall be not more than one delegate for a given bundle version
(parallel channels) a delegate shall not delegate another delegate
(completeness) an unknown id shall fail graph metadata validation
OLM v0 GRPC APIs used
- pkg/controller/registry/registry_client.go
- GetLatestChannelEntriesThatProvide
- GetBundle
- GetPackage
- pkg/package-server/provider/registry.go
- GetBundleForChannel (*deprecated*)
- ListBundles
- ListPackages
- GetPackage
v0 RFC says:
> The bundle objects provided by the registry v1 GRPC API must include the deprecation properties at minimum for the methods of interest to the resolver(ListBundles) and package-server(GetBundleForChannel)
## DEMO Info
### GRPC
catalog docker.io/grokspawn/deprecation-catalog:latest
demos kiali-operator with a deprecated bundle/channel/package.
Show the whole yaml structure with deprecations in-place:
```shell=
opm render docker.io/grokspawn/deprecation-catalog:latest -o yaml | yq 'select(.name == "kiali" or .package == "kiali")'
```
(long output)
Show just the deprecations (showing like a real catalog, even though could just dump the single deprecations instance):
```shell=
opm render docker.io/grokspawn/deprecation-catalog:latest -o yaml | yq 'select((.name == "kiali" or .package == "kiali") and .schema == "olm.deprecations")'
```
output:
```yaml=
entries:
- message: |
kiali-operator.v1.68.0 is deprecated. Uninstall and install kiali-operator.v1.72.0 for support.
reference:
name: kiali-operator.v1.68.0
schema: olm.bundle
- message: |
package kiali is end of life. Please use 'kiali-new' package for support.
reference:
schema: olm.package
- message: |
channel alpha is no longer supported. Please switch to channel 'stable'.
reference:
name: alpha
schema: olm.channel
package: kiali
schema: olm.deprecations
```
Run a catalog instance
```shell=
docker run --rm -it -p 50051:50051 docker.io/grokspawn/deprecation-catalog:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
WARN[0000] unable to set termination log path error="open /dev/termination-log: permission denied"
INFO[0000] starting pprof endpoint address="localhost:6060"
INFO[0000] serving registry configs=/configs port=50051
INFO[0000] stopped caching cpu profile data address="localhost:6060"
```
Interact with the GRPC `ListBundles` API
```shell=
grpcurl -plaintext localhost:50051 api.Registry/ListBundles | jq 'select(.packageName == "kiali" and .csvName == "kiali-operator.v1.68.0")'
```
```json=
{
"csvName": "kiali-operator.v1.68.0",
"packageName": "kiali",
"channelName": "stable",
"bundlePath": "quay.io/openshift-community-operators/kiali@sha256:61e8f5a82463537409341e6c8886aaf2d43ba360af92e90a1837cd2bccf1580a",
"providedApis": [
{
"group": "kiali.io",
"version": "v1alpha1",
"kind": "Kiali"
}
],
"version": "1.68.0",
"skipRange": ">=1.0.0 <1.68.0",
"properties": [
{
"type": "olm.gvk",
"value": "{\"group\":\"kiali.io\",\"kind\":\"Kiali\",\"version\":\"v1alpha1\"}"
},
{
"type": "olm.package",
"value": "{\"packageName\":\"kiali\",\"version\":\"1.68.0\"}"
}
],
"replaces": "kiali-operator.v1.67.0",
"deprecation": {
"message": "kiali-operator.v1.68.0 is deprecated. Uninstall and install kiali-operator.v1.72.0 for support.\n"
}
}
```
Hey... look.... deprecations :smile:
`ListPackages`
```shell=
grpcurl -plaintext localhost:50051 api.Registry/ListPackages | jq '.name'| grep -i -A 5 -B 5 kiali
```
output
```shell=
"group-sync-operator"
"gitops-primer"
"shipwright-operator"
"apimatic-kubernetes-operator"
"machine-deletion-remediation"
"kiali"
"postgresql"
"ack-kms-controller"
"cockroachdb"
"eclipse-che"
"ack-sqs-controller"
```
`GetBundle`
```shell=
grpcurl -plaintext -d '{"pkgName": "kiali", "channelName": "alpha", "csvName": "kiali-operator.v1.68.0"}' localhost:50051 api.Registry/GetBundle
```
```json=
{
"csvName": "kiali-operator.v1.68.0",
"packageName": "kiali",
"channelName": "alpha",
"bundlePath": "quay.io/openshift-community-operators/kiali@sha256:61e8f5a82463537409341e6c8886aaf2d43ba360af92e90a1837cd2bccf1580a",
"providedApis": [
{
"group": "kiali.io",
"version": "v1alpha1",
"kind": "Kiali"
}
],
"version": "1.68.0",
"skipRange": "\u003e=1.0.0 \u003c1.68.0",
"properties": [
{
"type": "olm.gvk",
"value": "{\"group\":\"kiali.io\",\"kind\":\"Kiali\",\"version\":\"v1alpha1\"}"
},
{
"type": "olm.package",
"value": "{\"packageName\":\"kiali\",\"version\":\"1.68.0\"}"
}
],
"deprecation": {
"message": "kiali-operator.v1.68.0 is deprecated. Uninstall and install kiali-operator.v1.72.0 for support.\n"
}
}
```
We know that 1.68.0 is deprecated, but it also replaces 1.67. Grab it again but via `GetBundleThatReplaces`
```shell=
grpcurl -plaintext -d '{"pkgName": "kiali", "channelName": "alpha", "csvName": "kiali-operator.v1.67.0"}' localhost:50051 api.Registry/GetBundleThatReplaces
```
```json=
{
"csvName": "kiali-operator.v1.68.0",
"packageName": "kiali",
"channelName": "alpha",
"bundlePath": "quay.io/openshift-community-operators/kiali@sha256:61e8f5a82463537409341e6c8886aaf2d43ba360af92e90a1837cd2bccf1580a",
"providedApis": [
{
"group": "kiali.io",
"version": "v1alpha1",
"kind": "Kiali"
}
],
"version": "1.68.0",
"skipRange": "\u003e=1.0.0 \u003c1.68.0",
"properties": [
{
"type": "olm.gvk",
"value": "{\"group\":\"kiali.io\",\"kind\":\"Kiali\",\"version\":\"v1alpha1\"}"
},
{
"type": "olm.package",
"value": "{\"packageName\":\"kiali\",\"version\":\"1.68.0\"}"
}
],
"deprecation": {
"message": "kiali-operator.v1.68.0 is deprecated. Uninstall and install kiali-operator.v1.72.0 for support.\n"
}
}
```
### PackageServer Demo
https://asciinema.org/a/xlDhS5xd8ARTqbJ9MVboIPIlK
Sign in with Wallet
Connect another wallet