# Operand Versioning / Image Selection

## Problem

Operators that manage N versions of an operand need a way to map a supported operand version to an image reference. Ideally this mapping could be updated out of band from the operator itself, i.e. shipping CVE fixes to operands without shipping a new operator version.

## Goals

- One or more recommended options to give to teams with operand image versioning needs.
- At least one option that works disconnected
- Clear for any given option what is and isn't supported (i.e. disconnected, proxy, etc)
- A set of scoped work to deliver on the most valuable options

## Today

This problem is solved today via `related images` and automation which ensures up-to-date operand image values are available to the operator as environment variables.

Operator pipelines do image name substitution to allow the operator to get updated versions at runtime.

For example, a CSV might define:

kind: ClusterServiceVersion 
metadata:
  name: etcd-operator
spec:
  relatedImages:
  - name: default
    image: quay.io/coreos/etcd:stable 
  - name: etcd-2.1.5
    image: quay.io/coreos/etcd@sha256:12345 
  - name: etcd-3.1.1
    image: quay.io/coreos/etcd@sha256:12345 
  install:
    strategy: deployment
    spec:
      deployments:
      - name: etcd-operator
        spec:
          replicas: 1
          selector:
            matchLabels:
              app: etcd-operator
          template:
            metadata:
              labels:
                app: etcd-operator
            spec:
              containers:
              - name: etcdop
                image: etcd-operator:1.1
                ports:
                - containerPort: 80
                env:
                - name: ETCD_DEFAULT
                  value: quay.io/coreos/etcd:stable 
                - name: ETCD_V2
                  value: quay.io/coreos/etcd@sha256:12345 
                - name: ETCD_V3
                  value: quay.io/coreos/etcd@sha256:12345 

The pipeline will resolve tags and pin versions to digests before releasing the operator. At runtime, the operator can read the latest values from the environment. This is similar to the image replacement that is used in the OCP release payload.

For more info, see the [digest pinning docs](https://docs.engineering.redhat.com/display/CFC/Digest_Pinning).

Pros:
- Works
- Available today
- Strong provenance (operator with digest XYZ only uses operand image with digest ABC)
- Disconnected workflow picks up related images for mirroring

Cons:
- Only available to operators shipping through red hat pipelines
- No way to update values without releasing/respinning a new version of the operator
- Annoying with large N

## Tomorrow (after 4.8)

The declarative index config + pipeline support for it will allow releasing new related images and a new "entry" in the catalog without spinning a new operator bundle + errata.

## Strawman Options

### Related Image Catalog Entry

The [declarative index config](https://github.com/operator-framework/enhancements/pull/37/files) allows for multiple artifact types to be referenced in a catalog.

Define a related image catalog entry

{
    "schema": "olm.relatedimages",
    "name": "etcdoperator-related",
    "version": "0.9.0",
    "relatedImages" : [
        {
            "name": "etcd-stable",
            "image": "quay.io/coreos/etcd-operator@sha256:db563baa8194fcfe39d1df744ed70024b0f1f9e9b55b5923c2f3a413c44dc6b8"
        }
    ]
}

This can be referenced via "bundle" entries that exist today, by defining a new dependency type:

{
    "schema": "olm.bundle", 
    "name": "etcdoperator-community.v0.6.1",
    "package": "etcd",
    "image": "quay.io/operatorhubio/etcd:v0.6.1",
    "version": "0.6.1",
    "properties":[
        {
            "name": "olm.relatedImageBundle",
            "value": {
                "name": "etcdoperator-related",
                "version": ">= 0.9.0 < 1.0.0"
            },
        },
        // rest omitted 
    ]
}

The resolver will pull and install the `etcdoperator-related` bundle to the cluster and keep it up to date. The related image bundle can be updated independently from the operator, and can even be shipped in a separate catalog. The operator will have some contract with related image bundles (i.e. knows to look in a particular configmap).

Hand-waving implementation, because it will be heavily influenced by the declarative index config, and how far along OLM is with the v1 APIs.

### Operand Bundle / Catalog Entry

Model operands as a first-class concept in the catalog.

Define an operand bundle / catalog entry.

{
    "schema": "olm.operand",
    "name": "etcd",
    "version": "3.1.1",
    "images" : [
        {
            "name": "etcd",
            "image": "quay.io/coreos/etcd@sha256:db563baa8194fcfe39d1df744ed70024b0f1f9e9b55b5923c2f3a413c44dc6b8"
        }
    ]
}

An operator bundle can reference them:

{
    "schema": "olm.bundle", 
    "name": "etcdoperator-community.v0.6.1",
    "package": "etcd",
    "image": "quay.io/operatorhubio/etcd:v0.6.1",
    "version": "0.6.1",
    "properties":[
        // supports 2.1.0-3.0.0 and 3.1.1 etcd
        {
            "name": "olm.operand.supported",
            "value": {
                "name": "etcd",
                "version": "3.1.1"
            },
        },
        {
            "name": "olm.operand.supported",
            "value": {
                "name": "etcd",
                "version": ">= 2.1.0 < 3.0.0"
            },
        },
        // requires an operator the provides the prometheus operand
        {
            "name": "olm.operand.required",
            "value": {
                "name": "prometheus",
                "version": ">= 2.1.0 < 3.0.0"
            },
        },
        // rest omitted 
    ]
}

Operators can provide or depend on operands much in the same way they can provide or depend on APIs. The difference in this model is that an operand also corresponds to a definition that includes a list of bundles.

This would allow for disconnected mirroring to restrict the set of images required for airgapped installation to only those that will be used, if the user provides the information up-front about required operands.

The resolver will pull and install the `etcd 3.1.1` operand bundle to the cluster and keep it up to date. The operand bundle can be updated independently from the operator, and can even be shipped in a separate catalog. The operator will have some contract with related image bundles (i.e. knows to look in a particular configmap).

Because the operand image has its own identity in a catalog, we may find we need to associate them with particular packages or operator versions to avoid conflicts.

### Cincinatti

Example Cincinatti data:

{
    "nodes":
    [{
        "version":"0.0.0",
        "payload":"quay.io/redhat/openshift-cincinnati-test-public-manual@sha256:a264db3ac5288c9903dc3db269fca03a0b122fe4af80b57fc5087b329995013d",
        "metadata":{
            "io.openshift.upgrades.graph.release.channels":"a",
            "io.openshift.upgrades.graph.release.manifestref":"sha256:a264db3ac5288c9903dc3db269fca03a0b122fe4af80b57fc5087b329995013d"
        }
    },
    {
        "version":"0.0.1",
        "payload":"quay.io/redhat/openshift-cincinnati-test-public-manual@sha256:73df5efa869eaf57d4125f7655e05e1a72b59d05e55fea06d3701ea5b59234ff",
        "metadata":{
            "io.openshift.upgrades.graph.release.manifestref":"sha256:73df5efa869eaf57d4125f7655e05e1a72b59d05e55fea06d3701ea5b59234ff",
            "io.openshift.upgrades.graph.release.channels":"a",
            "kind":"test"
        }
    }],
    "edges":[[0,1]]
}

The operator may include configuration to watch a specific cincinatti endpoint, which provides updated version -> image mappings.

We may define a Cincinati Graph bundle type to add to an index, which operators may depend on as in [Related Image Catalog Entry
](#Related-Image-Catalog-Entry)

Instead of the operator knowing to look in a specific location (e.g. configmap), the operator knows to talk to a certain endpoint.

### Downward API

The [related images](https://github.com/operator-framework/operator-lifecycle-manager/blob/master/doc/contributors/design-proposals/related-images.md#future-work) proposal sketches out future work that would make related images automatically available as annotations on the operator deployments. This allows authors to get the values via the downward API at runtime.

This does not address the bulk of the problem, but would open up the env var updating feature to users outside of red hat pipelines. It could also be a building block for other options at runtime.

### Tags

Tags decouple image content from the name, but lack provenance.

### Manifest lists

An operator may depend on a manifestlist, which is already a list of images.

This won't be effective unless OCI images can be fully supported by red hat pipelines. Docker v2-2 images restrict manifestlists to arch variants only.

### Other Options

TBD