FBC / catalog templates 101

(Note: catalog templates had a working title of veneers. We're attempting to consistently rename, but this should at least demystify any missed references.)

Background

File-Based Catalogs (FBC) came about because (some reasons):

• implicit graph generation incompatibilities (replaces vs. skips) for new package add
• opacity of index generation/maintenance
• forcing OLM to be a cog in the middle of the 'catalog manipulation machine'

More details in the how we got to catalog templates and even in announcement of FBC on the operator-framework g-group.

FBCs are the foundational building block for other projects, like KCP, RukPak, and PlatformOperators. But … they can get complex and full of meta-data we can retrieve once a bundle has been deployed. Is there an easier way? Enter …

Catalog Templates

A catalog template (hereafter: template) is a simplification over directly dealing with FBC.

The single defining characteristic of a template is that when it is processed it will always yield valid FBC. This means that there are many approaches, with only one valid result.

The tool that is used to process these templates into FBC is not material. But there is a lot of potential benefit for RH to set expectations & establish good ground rules by providing examples. There is also a bunch of benefit to 'officially supported' veneers as a message of goodwill to operator authors and a promise of continued support.

In the spirit of the 80/20 rule, we set out to implement three template types to capture primordial use-cases:

1. basic template captures "how can we have an FBC, but simpler to maintain?"
2. semver template captures "how can I express channel promotion by leveraging common semver concepts?"
3. composite template captures "how can I coordinate the processing of multiple veneers or multiple destination catalogs?"

We'll cover each of these briefly, relying mainly on existing documentation

Basic Template

The PR has really good documentation for this in detail, as well as the design doc and the demo.

In essence though, this template is a complete FBC that allows the operator author the ability to skip all of the olm.bundle details. This separates the actions of "release the bundle" from "generate the catalog", and it also simplifies the maintenance efforts since it can easily reduce a full FBC down to a representative 20 lines or so.

Operator catalog generation becomes

1. release bundle
2. update the basic template to point to the new tagged release of the bundle
3. generate an FBC from the template

Yay, simple. But operators want to do more than this. They want to express complex notions of sequentiality over a range of versions of an operator, and make the complex … simple. Semver gives us a lever for deterministic sequencing of things, which leads to …

SemVer template

The PR has really good documentation of how this work, including examples. There is also a README.md (and in that directory are also asciinema GIFs of evaluating both the minor and major channel generation approaches) and a hugely complex and descriptive design doc. Please read the README.md and design doc for more of the details of this veneer.

This template allows an operator author to trivially promote a bundle into channel(s) of their choice, and trivially maintain the resulting info.

And one of the coolest things is that you can output the channel edges in mermaid, so you can view (and debug) them graphically, for e.g.

graph LR
testoperator.v0.1.0
testoperator.v0.1.1
testoperator.v0.1.2
testoperator.v0.1.3
testoperator.v0.1.3 -- skips --> testoperator.v0.1.0
testoperator.v0.1.3 -- skips --> testoperator.v0.1.1
testoperator.v0.1.3 -- skips --> testoperator.v0.1.2
testoperator.v0.2.0
testoperator.v0.2.1
testoperator.v0.2.2
testoperator.v0.2.2 -- replaces --> testoperator.v0.1.3
testoperator.v0.2.2 -- skips --> testoperator.v0.2.0
testoperator.v0.2.2 -- skips --> testoperator.v0.2.1
testoperator.v0.3.0
testoperator.v0.3.0 -- replaces --> testoperator.v0.2.2

Between the basic and semver template, we feel like we've covered the most common operator author use-cases. But we haven't addressed the scaling question for managing multiple template, for which we need something like

VeneerGroup Template

The design doc for this establishes a good basis for this, the intent being to allow an individual operator author with the tools necessary to handle how templates are processed into FBCs with arbitrary granularity, so long as they pass opm validate test on the results. Do they wish to process a different template with a different OPM binary per release? Process the same template with discrete OPM binaries for different release versions? Contribute the same generated FBC to multiple catalogs?

After looking into alternate implementations of a veneergroups without the use of specially built binaries for the purpose, it seems fairly straightforward to use existing tools for doing the same. This demo, using the repository linked here, shows how make can do what the veneergroup design doc outlines cleanly.

Composite Template

After determining that there were adequate tools to approach a one-to-many publishing model for FBC, we discovered that the most significant gap in the operator-author <–> catalog maintainer relationship was clear end-to-end communication of catalog submission requirements (for e.g. opm version, accepted templates/formats).

That is, communications from author to catalog were trivial, but catalog to author were missing.

Note: official docs don't yet exist, but FBC enablement - Community and FBC Enablement - HACBS include discussion on the composite template.

Timeline of Contributing Sources

Demo / presentation list