file-based-catalog-veneers

Release Signoff Checklist

• Enhancement is implementable
• Design details are appropriately documented from clear requirements
• Test plan is defined
• Graduation criteria for dev preview, tech preview, GA

Open Questions

• How much support should be provided for installation and discovery of veneers?
• How should Freshmaker support work? Given extensibility goals for veneers, it seems obvious that Freshmaker will not be able to hook into veneers. Therefore, it seems like we need Freshmaker to hook in as a post-processing step, always operating on the output of the veneer rendering. See https://github.com/operator-framework/operator-registry/issues/782#issuecomment-931411205.
• Do we need a veneer for stripping fields from objects, (e.g: olm.bundle.object from non-head bundles)?
• what expressiveness is lost when using a veneer vs the olm.channel schema? Should our channel veneers support extra static entries? If we do that, how would we hook the generated entries into the extra static entries?

Summary

This enhancement proposes a minimal set of veneers that operator authors can use to simplify maintaining file based catalogs. It also describes an overview of what workflows using veneers may look like.

Motivation

File based catalogs place all information about bundles, channels and packages in one or more flat files that can be directly manipulated by operator authors. This however exposes a lot of information that authors did not have to concern themselves with the imperative workflow followed by opm with sqlite based catalogs.

Veneers aim at simplifying the file based catalog generation for operator authors, presenting a set of veneers that may be expanded to get a valid file based catalog while reducing the configuration that needs to be done to obtain a valid file based catalog.

Goals

File based catalogs allows for operator authors to declaratively define metadata surrounding their operator, including specific properties and upgrade behavior. In the process of exposing this information, it also exposes a lot of complexity that they then have to deal with, making publishing an operator difficult. This enhancement proposes a API to allow interacting with a simplified view or veneer, containing only subset of information provided by the whole file based catalog.

• Composability:
  • As an operator author, I need to be able to use multiple veneers simultaneously.
  • As an operator author, I need to be able to use veneers and low-level olm.package, olm.channel, and olm.bundle blobs simultaneously.
• Extensibility:
  • As an operator author, I need to be able to define a custom veneer API and develop tooling that can expand my custom veneer to the low-level formats.

Non-Goals

• Bundle modification:
  • Veneers discussed in this proposal are primarily to define package and channel membership for bundles. Modifying the bundle or bundle metadata apart from properties are not in the scope of the enhancement.
• Idempotency:
  • Applying a set of veneers out of order or applying a veneer to an already processed file based catalog might not necessarily produce the same results even if the same set of veneers are applied

Proposal

This enhancement describes:

1. The expected format and workflow for veneers
2. A set of official veneers maintained by opm

The Veneer API

A veneer is a binary that can reduce a file based catalog into a subset of its fields and override specific fields on a catalog. Any executable can act as a veneer as long as it respects the following restrictions:

1. Input:
   • Zero or more input paths to a directory/file containing a partial or complete file based catalog
   • Any number of optional arguments
2. Output:
   • A partial or complete file based catalog in a recognizable format (YAML/JSON), preserving the schema and package information, along with any source information like the image of a bundle
   • Must be writeable to a target directory, which may be '-' for stdout, with modified files having the same name as the parent file they were applied on
   • A history file, either created anew or containing a list of veneer calls applied so far with the most recent call appended to it.

Each veneer need not create a catalog that is valid. The final output of applying a series of veneers is completed by the command opm inflate, which fills in any required and missing fields from a data source like an image.

The veneers exist outside of the catalog itself and can be ignored with .indexignore entries. An operator using veneers may have its repository structured as below:

.
├── awesome-operator
│   ├── .indexignore
│   ├── bundle-1.yaml
│   ├── bundle-2.yaml
│   ├── bundle-3.yaml
│   ├── channels.yaml
│   └── package.yaml
└── veneer
    ├── awesome-operator
    │   └── properties.config.yaml
    ├── olm.properties.veneer
    ├── olm.channel.veneer
    ├── my.awesome.veneer
    └── veneer.history

The veneers may then applied like:

./veneer/olm.properties.veneer -o ./awesome-operator/ ./awesome-operator/properties.yaml --config ./veneer/awesome-operator/properties.config.yaml
./veneer/olm.channel.veneer -o ./awesome-operator/ ./awesome-operator/channels.yaml
./veneer/my.awesome.veneer ./awesome-operator/bundle-*  -o - > ./awesome-operator/bundles.yaml
opm inflate ./awesome-operator > catalog.yaml

For a catalog structure as show above, applying a set of veneers looks like:

$ ./veneer/olm.properties.veneer ./awesome-operator/ -i --config=config/properties.yaml
$ ./veneer/olm.channel.veneer ./awesome-operator/channels.yaml -o ./awesome-operator/channels.foo.yaml
$ ./veneer/my.awesome.veneer --bar=baz ./awesome-operator -o - > final-catalog.yaml  

operator-framework supported veneers

There are some commonly encountered operations on file based catalogs that operator-framework will provide veneers for. Apart from these officially supported veneers, operator authors may use third party veneers or create their own veneers for specific use cases. olm supported veneers are additive: each veneer ensures certain fields on their input.

The two types of veneers supported by operator-framework are:

Package/channel membership veneers

These are meant for adding bundles to packages and channels and to define specific upgrade edges between bundle versions. Veneers in this set ensure specific upgrade edges on a channel. The olm channel veneers do not generate skipRange entries, so some post processing is required in cases where this is needed.

The following officially supported veneers fall into this category:

Semver veneer

This veneer allows a channel to be populated with a set of bundles in semver order. The generated upgrade chain will have no skipped bundles unless skipPatch is set to true. This veneer takes the following arguments:

1. package: The package a bundle belongs to. Each bundle must belong to exactly one package
2. name: The channel name to be generated, which may be a template with semver information
3. semverRange: The valid semver range for bundles to be included. All bundles in the provided catalog directory belonging to the specified package will be evaluated.
4. skipPatch: Optionally enable skipPatch ordering, i.e. lower patch versions will be added as skips edges for the greatest patch version of any <major>.<minor> version set.
   Bundles added to a channel will be arranged by their semver, with each bundle replacing the preceeding version. Skips edges are only added in skipPatch mode.

A sample configuration for this veneer might look like:

{
    "schema":"olm.veneer.semver",
    "package":"awesome-operator",
    "name":"release-v{{.Major}}.{{.Minor}}",
    "semverRange":">={{.Major}}.{{.Minor}}.x",
    "skipPatch": true
}

When applied to a catalog like below, an olm.channel object is generated.

.
└── awesome-operator
    ├── .indexignore
    ├── bundle.v1.0.0.yaml
    ├── bundle.v1.0.1.yaml
    ├── bundle-v1.0.2.yaml
    ├── bundle-v2.0.0.yaml
    └── package.yaml

{
    "schema": "olm.channel",
    "name": "release-v1.0",
    "package": "awesome-operator",
    "entries": [
        {
            "name": "bundle.v1.0.0",
        },
        {
            "name": "bundle.v1.0.1",
            "replaces": "bundle.v1.0.0"
        },
        {
            "name": "bundle.v1.0.2",
            "skips": [
                "bundle.v1.0.0",
                "bundle.v1.0.1"
            ]
        }
    ]
}
{
    "schema": "olm.channel",
    "name": "release-v2.0",
    "package": "awesome-operator",
    "entries": [
        {
            "name": "bundle.v2.0.0"
        }
    ]
}

Covington

Covington (named for its proximity to Cincinnati) is a veneer that builds a channel based on a defined order of versions in a channel. It creates an upgrade chain out of a provided list of bundles.
The config for Covington has the following fields:

1. package: The package a bundle belongs to. Each bundle must belong to exactly one package.
2. name: The channel name to be generated, which may be a template with semver information.
3. entries: The list of bundles to be added to the channel in order of the upgrade path. The upgrade chain is synthesised preserving this topological ordering,
4. tombstones: Entries to be skipped in the upgrade chain and not considered for installation or upgrade.
5. tailEdges: Entries to be skipped on the tail bundle. This allows connecting the upgrade chain to an existing portion of the chain.

This veneer forms a replaces chain from non-tombstoned versions. The basic rules are:

1. Only non-tombstoned entries result in entries in the channel
2. Each non-tombstoned entry will replace the immediately preceding non-tombstoned entry
3. Each non-tombstoned entry will skip all tombstoned entries between it and the next immediately preceding non-tombstoned entry.
4. The tail of the generated channel will include tailEdges in its entry.
5. The last item in the entries array must not be in the tombstones array.

A sample config for this looks like:

{
    "schema": "olm.veneer.covington",
    "package": "awesome-operator",
    "name": "release-v1.0",
    "entries": [
        "example.v0.1.0",
        "example.v0.2.0",
        "example.v0.2.1",
        "example.v0.2.2",
        "example.v1.0.0",
        "example.v1.0.1"
    ],
    "tombstones": [
        "example.v0.2.0",
        "example.v0.2.1",
        "example.v1.0.0"
    ],
    "tailEdges": {
        "skips": [
            "example.v0.0.1",
            "example.v0.0.2"
        ]
    }
}

When applied, the above config generates an olm.channel object like:

{
    "schema": "olm.channel",
    "name": "release-v1.0",
    "package": "awesome-operator",
    "entries": [
        {
            "name": "example.v0.1.0",
            "replaces": "example.v0.1.0",
            "skips": [
                "example.v0.0.1",
                "example.v0.0.2"
            ]
        },
        {
            "name": "example.v0.2.2",
            "replaces": "example.v0.1.0",
            "skips": [
                "example.v0.2.0",
                "example.v0.2.1"
            ]
        },
        {
            "name": "example.v1.0.1",
            "replaces": "example.v0.2.2",
            "skips": [
                "example.v1.0.0"
            ]
        }
    ]   
}

Upgrade edge veneer

The upgrade edge veneer allows explicitly specifying edges to be included in a channel.
The veneer's config accepts the following arguments:

1. package: The package the channel belongs to
2. name: The name of the channel to be generated
3. tombstones: a list of bundles to be skipped in an upgrade chain. Any bundle specified here will be added as a skips edge if present as a <from bundle> in the upgradeEdges field
4. upgradeEdges: a list of valid upgrade edges in the form of [<from bundle>, <to bundle>]. In case of multiple edges to the same bundle, the first non-tombstone <from_bundle> will be a replaces entry and the rest will be added as skips entries on the <to bundle>.

A sample config file for this veneer would look like:

{
    "schema": "olm.veneer.upgrade",
    "package": "awesome-operator",
    "name": "release-v1.0",
    "tombstones": [
        "awesome-operator.v1.0.0"
    ],
    "upgradeEdges": [
        ["awesome-operator.v1.0.0", "awesome-operator.v1.0.1"],
        ["awesome-operator.v1.0.1", "awesome-operator.v1.0.2"],
        ["awesome-operator.v1.0.0", "awesome-operator.v1.0.3"],
        ["awesome-operator.v1.0.2", "awesome-operator.v1.0.3"],
        ["awesome-operator.v1.0.1", "awesome-operator.v1.0.3"],
    ]
}

Applying the above config results in an olm.channel object like:

{
    "schema": "olm.channel",
    "name": "release-v1.0",
    "package": "awesome-operator",
    "entries": [
        {
            "name": "awesome-operator.v1.0.1",
            "skips": [
                "awesome-operator.v1.0.0"
            ]
        },
        {
            "name": "awesome-operator.v1.0.2",
            "replaces": "awesome-operator.v1.0.1",
            "skips": [
                "awesome-operator.v1.0.0"
            ]
        },
        {
            "name": "bundle.v1.0.3",
            "replaces": "awesome-operator.v1.0.2",
            "skips": [
                "bundle.v1.0.0",
                "bundle.v1.0.1"
            ]
        }
    ]   
}

Metadata modification veneers

These are meant for adding or modifying metadata to objects. The bundle override veneer described below belongs to this type of veneers.

Bundle override veneer

This veneer allows ensuring or overriding specific fields on a bundle. It searches for an object with a matching schema, package and name, ensuring members of the override field on the bundle. Any property entries are appended to the existing list of properties while relatedImages are overridden based on the name.

It accepts the following arguments:

1. name: Name of the rendered bundle to modify. The olm.bundle object must already be present in the input catalog directory, or a stub olm.bundle object with only the specified properties will be generated.
2. package: The package a bundle belongs to. This is used along with name to identify the olm.bundle object to modify.
3. override: A set of values to be ensured on the olm.bundle object. It supports overriding properties and relatedImages, but may be extended to support arbitrary fields.
   3.1. properties: These are properties that must be ensured on the bundle object. Property objects can have multiple entries for each type.
   3.2. relatedImages: These are overrides to the relatedImages field in a bundle object. Overrides allow only one value per name for this field, so a value may be overwritten.

A sample config file for the veneer looks like:

{
    "schema": "olm.veneer.bundle",
    "name": "my-awesome-operator",
    "package": "awesome-package",
    "override": {
        "properties": [
            {
                "type": "olm.maxOpenShiftVersion",
                "value": "4.8"
            },
            {
                "type": "my.awesome.property",
                "value": "foo"
            }
        ],
        "relatedImages": [
            {
                "name": "",
                "image": "registry.redhat.io/awesome/new@sha256:b480e64de68bbe7c271623cf9a9f88067fa7d1dd4dc54f192ca505ecfb25dd17"
            }
        ]
    }
}

This can then be applied to a bundle object like below:

{
    "schema": "olm.bundle",
    "name": "my-awesome-operator",
    "package": "awesome-package",
    "image": "registry.redhat.io/awesome/operator@sha256:55166d313d270ae4740a260149bfb41df1af8214744e9c38abef07add49c95de",
    "properties": [
        {
            "type": "olm.gvk",
            "value": {
                "group": "api.awesome.org",
                "kind": "APIManager",
                "version": "v1alpha1"
            }
        },
        {
            "type": "my.awesome.property",
            "value": "bar"
        },
        {
            "type": "olm.bundle.object",
            "value": {
                "data": "eyJhcGlWZXJzaW9uIjoiYXBpZXh0ZW5zaW9ucy5rOHMuaW8vdjFiZXRhMSIsImtpbmQiOiJDdXN0b21SZXNvdXJjZURlZmluaXRpb24iLCJtZXRhZGF0YSI6eyJuYW1lIjoibGltaXRzLmNhcGFiaWxpdGllcy4zc2NhbGUubmV0In0sInNwZWMiOnsiZ3JvdXAiOiJjYXBhYmlsaXRpZXMuM3NjYWxlLm5ldCIsIm5hbWVzIjp7ImtpbmQiOiJMaW1pdCIsImxpc3RLaW5kIjoiTGltaXRMaXN0IiwicGx1cmFsIjoibGltaXRzIiwic2luZ3VsYXIiOiJsaW1pdCJ9LCJzY29wZSI6Ik5hbWVzcGFjZWQiLCJzdWJyZXNvdXJjZXMiOnsic3RhdHVzIjp7fX0sInZhbGlkYXRpb24iOnsib3BlbkFQSVYzU2NoZW1hIjp7InByb3BlcnRpZXMiOnsic3BlYyI6eyJwcm9wZXJ0aWVzIjp7Im1heFZhbHVlIjp7ImZvcm1hdCI6ImludDY0IiwidHlwZSI6ImludGVnZXIifSwibWV0cmljUmVmIjp7InByb3BlcnRpZXMiOnsiYXBpVmVyc2lvbiI6eyJ0eXBlIjoic3RyaW5nIn0sImZpZWxkUGF0aCI6eyJ0eXBlIjoic3RyaW5nIn0sImtpbmQiOnsidHlwZSI6InN0cmluZyJ9LCJuYW1lIjp7InR5cGUiOiJzdHJpbmcifSwibmFtZXNwYWNlIjp7InR5cGUiOiJzdHJpbmcifSwicmVzb3VyY2VWZXJzaW9uIjp7InR5cGUiOiJzdHJpbmcifSwidWlkIjp7InR5cGUiOiJzdHJpbmcifX0sInR5cGUiOiJvYmplY3QifSwicGVyaW9kIjp7InR5cGUiOiJzdHJpbmcifX0sInJlcXVpcmVkIjpbInBlcmlvZCIsIm1heFZhbHVlIiwibWV0cmljUmVmIl0sInR5cGUiOiJvYmplY3QifSwic3RhdHVzIjp7InR5cGUiOiJvYmplY3QifX19fSwidmVyc2lvbiI6InYxYWxwaGExIiwidmVyc2lvbnMiOlt7Im5hbWUiOiJ2MWFscGhhMSIsInNlcnZlZCI6dHJ1ZSwic3RvcmFnZSI6dHJ1ZX1dfX0="
            }
        },
        {
            "type": "olm.bundle.object",
            "value": {
                "data": "eyJhcGlWZXJzaW9uIjoiYXBpZXh0ZW5zaW9ucy5rOHMuaW8vdjFiZXRhMSIsImtpbmQiOiJDdXN0b21SZXNvdXJjZURlZmluaXRpb24iLCJtZXRhZGF0YSI6eyJuYW1lIjoibWFwcGluZ3J1bGVzLmNhcGFiaWxpdGllcy4zc2NhbGUubmV0In0sInNwZWMiOnsiZ3JvdXAiOiJjYXBhYmlsaXRpZXMuM3NjYWxlLm5ldCIsIm5hbWVzIjp7ImtpbmQiOiJNYXBwaW5nUnVsZSIsImxpc3RLaW5kIjoiTWFwcGluZ1J1bGVMaXN0IiwicGx1cmFsIjoibWFwcGluZ3J1bGVzIiwic2luZ3VsYXIiOiJtYXBwaW5ncnVsZSJ9LCJzY29wZSI6Ik5hbWVzcGFjZWQiLCJzdWJyZXNvdXJjZXMiOnsic3RhdHVzIjp7fX0sInZhbGlkYXRpb24iOnsib3BlbkFQSVYzU2NoZW1hIjp7InByb3BlcnRpZXMiOnsic3BlYyI6eyJwcm9wZXJ0aWVzIjp7ImluY3JlbWVudCI6eyJmb3JtYXQiOiJpbnQ2NCIsInR5cGUiOiJpbnRlZ2VyIn0sIm1ldGhvZCI6eyJ0eXBlIjoic3RyaW5nIn0sIm1ldHJpY1JlZiI6eyJwcm9wZXJ0aWVzIjp7ImFwaVZlcnNpb24iOnsidHlwZSI6InN0cmluZyJ9LCJmaWVsZFBhdGgiOnsidHlwZSI6InN0cmluZyJ9LCJraW5kIjp7InR5cGUiOiJzdHJpbmcifSwibmFtZSI6eyJ0eXBlIjoic3RyaW5nIn0sIm5hbWVzcGFjZSI6eyJ0eXBlIjoic3RyaW5nIn0sInJlc291cmNlVmVyc2lvbiI6eyJ0eXBlIjoic3RyaW5nIn0sInVpZCI6eyJ0eXBlIjoic3RyaW5nIn19LCJ0eXBlIjoib2JqZWN0In0sInBhdGgiOnsidHlwZSI6InN0cmluZyJ9fSwicmVxdWlyZWQiOlsicGF0aCIsIm1ldGhvZCIsImluY3JlbWVudCIsIm1ldHJpY1JlZiJdLCJ0eXBlIjoib2JqZWN0In0sInN0YXR1cyI6eyJ0eXBlIjoib2JqZWN0In19fX0sInZlcnNpb24iOiJ2MWFscGhhMSIsInZlcnNpb25zIjpbeyJuYW1lIjoidjFhbHBoYTEiLCJzZXJ2ZWQiOnRydWUsInN0b3JhZ2UiOnRydWV9XX19"
            }
        }
    ],
    "relatedImages": [
        {
            "name": "",
            "image": "registry.redhat.io/awesome/old@sha256:55166d313d270ae4740a260149bfb41df1af8214744e9c38abef07add49c95de"
        }
        {
            "name": "awesome-operator",
            "image": "registry.redhat.io/awesome/operator-api@sha256:bb7dc249eb517b6e3ffd7d4ceb19884ad4af97f16835bf58e1930534ca7e4764"
        }
    ]
}

The output will look like:

{
    "schema": "olm.bundle",
    "name": "my-awesome-operator",
    "package": "awesome-package",
    "image": "registry.redhat.io/awesome/operator@sha256:55166d313d270ae4740a260149bfb41df1af8214744e9c38abef07add49c95de",
    "properties": [
        {
            "type": "olm.gvk",
            "value": {
                "group": "api.awesome.org",
                "kind": "APIManager",
                "version": "v1alpha1"
            }
        },
        {
            "type": "my.awesome.property",
            "value": "bar"
        },
        {
            "type": "olm.bundle.object",
            "value": {
                "data": "eyJhcGlWZXJzaW9uIjoiYXBpZXh0ZW5zaW9ucy5rOHMuaW8vdjFiZXRhMSIsImtpbmQiOiJDdXN0b21SZXNvdXJjZURlZmluaXRpb24iLCJtZXRhZGF0YSI6eyJuYW1lIjoibGltaXRzLmNhcGFiaWxpdGllcy4zc2NhbGUubmV0In0sInNwZWMiOnsiZ3JvdXAiOiJjYXBhYmlsaXRpZXMuM3NjYWxlLm5ldCIsIm5hbWVzIjp7ImtpbmQiOiJMaW1pdCIsImxpc3RLaW5kIjoiTGltaXRMaXN0IiwicGx1cmFsIjoibGltaXRzIiwic2luZ3VsYXIiOiJsaW1pdCJ9LCJzY29wZSI6Ik5hbWVzcGFjZWQiLCJzdWJyZXNvdXJjZXMiOnsic3RhdHVzIjp7fX0sInZhbGlkYXRpb24iOnsib3BlbkFQSVYzU2NoZW1hIjp7InByb3BlcnRpZXMiOnsic3BlYyI6eyJwcm9wZXJ0aWVzIjp7Im1heFZhbHVlIjp7ImZvcm1hdCI6ImludDY0IiwidHlwZSI6ImludGVnZXIifSwibWV0cmljUmVmIjp7InByb3BlcnRpZXMiOnsiYXBpVmVyc2lvbiI6eyJ0eXBlIjoic3RyaW5nIn0sImZpZWxkUGF0aCI6eyJ0eXBlIjoic3RyaW5nIn0sImtpbmQiOnsidHlwZSI6InN0cmluZyJ9LCJuYW1lIjp7InR5cGUiOiJzdHJpbmcifSwibmFtZXNwYWNlIjp7InR5cGUiOiJzdHJpbmcifSwicmVzb3VyY2VWZXJzaW9uIjp7InR5cGUiOiJzdHJpbmcifSwidWlkIjp7InR5cGUiOiJzdHJpbmcifX0sInR5cGUiOiJvYmplY3QifSwicGVyaW9kIjp7InR5cGUiOiJzdHJpbmcifX0sInJlcXVpcmVkIjpbInBlcmlvZCIsIm1heFZhbHVlIiwibWV0cmljUmVmIl0sInR5cGUiOiJvYmplY3QifSwic3RhdHVzIjp7InR5cGUiOiJvYmplY3QifX19fSwidmVyc2lvbiI6InYxYWxwaGExIiwidmVyc2lvbnMiOlt7Im5hbWUiOiJ2MWFscGhhMSIsInNlcnZlZCI6dHJ1ZSwic3RvcmFnZSI6dHJ1ZX1dfX0="
            }
        },
        {
            "type": "olm.bundle.object",
            "value": {
                "data": "eyJhcGlWZXJzaW9uIjoiYXBpZXh0ZW5zaW9ucy5rOHMuaW8vdjFiZXRhMSIsImtpbmQiOiJDdXN0b21SZXNvdXJjZURlZmluaXRpb24iLCJtZXRhZGF0YSI6eyJuYW1lIjoibWFwcGluZ3J1bGVzLmNhcGFiaWxpdGllcy4zc2NhbGUubmV0In0sInNwZWMiOnsiZ3JvdXAiOiJjYXBhYmlsaXRpZXMuM3NjYWxlLm5ldCIsIm5hbWVzIjp7ImtpbmQiOiJNYXBwaW5nUnVsZSIsImxpc3RLaW5kIjoiTWFwcGluZ1J1bGVMaXN0IiwicGx1cmFsIjoibWFwcGluZ3J1bGVzIiwic2luZ3VsYXIiOiJtYXBwaW5ncnVsZSJ9LCJzY29wZSI6Ik5hbWVzcGFjZWQiLCJzdWJyZXNvdXJjZXMiOnsic3RhdHVzIjp7fX0sInZhbGlkYXRpb24iOnsib3BlbkFQSVYzU2NoZW1hIjp7InByb3BlcnRpZXMiOnsic3BlYyI6eyJwcm9wZXJ0aWVzIjp7ImluY3JlbWVudCI6eyJmb3JtYXQiOiJpbnQ2NCIsInR5cGUiOiJpbnRlZ2VyIn0sIm1ldGhvZCI6eyJ0eXBlIjoic3RyaW5nIn0sIm1ldHJpY1JlZiI6eyJwcm9wZXJ0aWVzIjp7ImFwaVZlcnNpb24iOnsidHlwZSI6InN0cmluZyJ9LCJmaWVsZFBhdGgiOnsidHlwZSI6InN0cmluZyJ9LCJraW5kIjp7InR5cGUiOiJzdHJpbmcifSwibmFtZSI6eyJ0eXBlIjoic3RyaW5nIn0sIm5hbWVzcGFjZSI6eyJ0eXBlIjoic3RyaW5nIn0sInJlc291cmNlVmVyc2lvbiI6eyJ0eXBlIjoic3RyaW5nIn0sInVpZCI6eyJ0eXBlIjoic3RyaW5nIn19LCJ0eXBlIjoib2JqZWN0In0sInBhdGgiOnsidHlwZSI6InN0cmluZyJ9fSwicmVxdWlyZWQiOlsicGF0aCIsIm1ldGhvZCIsImluY3JlbWVudCIsIm1ldHJpY1JlZiJdLCJ0eXBlIjoib2JqZWN0In0sInN0YXR1cyI6eyJ0eXBlIjoib2JqZWN0In19fX0sInZlcnNpb24iOiJ2MWFscGhhMSIsInZlcnNpb25zIjpbeyJuYW1lIjoidjFhbHBoYTEiLCJzZXJ2ZWQiOnRydWUsInN0b3JhZ2UiOnRydWV9XX19"
            }
        },
        {
            "type": "olm.maxOpenShiftVersion",
            "value": "4.8"
        },
        {
            "type": "my.awesome.property",
            "value": "foo"
        }
    ],
    "relatedImages": [
        {
            "name": "",
            "image": "registry.redhat.io/awesome/new@sha256:b480e64de68bbe7c271623cf9a9f88067fa7d1dd4dc54f192ca505ecfb25dd17"
        }
        {
            "name": "awesome-operator",
            "image": "registry.redhat.io/awesome/operator-api@sha256:bb7dc249eb517b6e3ffd7d4ceb19884ad4af97f16835bf58e1930534ca7e4764"
        }
    ]
}

Using veneers

User Stories

As an operator author, I need to be able to use multiple veneers simultaneously.

A veneer must be able to accept another veneer's output for processing. This can be done be either having no strong restrictions on fields in the input catalog, or by specifying the expected fields on the input catalogs.

As an operator author, I need to be able to use veneers and low-level olm.package, olm.channel, and olm.bundle blobs simultaneously.

The file based catalog should not require that all bundles/packages use the same set of veneers. Each operator author should be free to have their set of veneers and their orders defined. Veneers are allowed to work on partial catalogs to generate required fields while leaving other parts untouched. Only the final catalog should be validated for correctness.

As an operator author, I need to be able to define a custom veneer API and develop tooling that can expand my custom veneer to the low-level formats.

Given a veneer and a custom api to expand it, opm must be able to use in its veneer expansion operation. For this purpose, the opm veneer expansion must be able to use external scripts/binaries with specified set of arguments to expand veneers.

As a catalog author, I should be able to use catalogs the same way regardless of how they are generated.

The final output of a set of veneers must be a usable catalog, performing the same as a low level file based catalog. To take the burden of additional support for veneer generated bundles, all veneer application should be done by the operator authors. The catalog authors and cluster administrators should only ever have to use the fully generated low level file based catalogs.

As an operator author, I need to be able to perform basic manipulation on file based catalogs for package/channel membership modification and editing mutable bundle properties

Operations like adding/removing a specific edge or node from an upgrade graph regardless of the position of the addition/removal must be easy for the operator author. This includes the ability to change channel membership.

Risks and Mitigations

Version Skew Strategy

To ensure veneers work with minor changes to the file based catalog schema, they are required to allow for unknown fields within an object's schema.

Alternatives

Split file based catalogs

While the veneer api allows easier maintenance of a file based catalog, many of its features can be obtained by simply rearranging objects in a file based catalog based on the package and object. This reduces the effort of parsing the entire catalog to make modifications.

└── awesome-operator
    ├── .indexignore
    ├── bundle.v1.0.0.yaml
    ├── bundle.v1.0.1.yaml
    ├── bundle-v1.0.2.yaml
    ├── bundle-v2.0.0.yaml
    ├── channel.alpha.yaml
    ├── channel.beta.yaml
    └── package.yaml

Ordering the package as above makes it easy to determine the portion of the catalog to be modified to add specific changes.

The addition of the olm.channel schema also makes it easier to manipulate a channel's upgrade edges compared to the prior channel membership model where it was required to declare channel membership on the bundle object. With this simplified schema, users have an easier time manipulating a package's upgrade paths.

The olm.channel object has the following schema: