---
tags: fbc,onboarding
title: FileBasedCatalogs - how we got to catalog templates
---

# FileBasedCatalogs - how we got to catalog templates

## Background:
previous approaches to catalogs used hidden SQLite databases to capture the info, which led to limitations, complications, and just doing things behind the curtain. FBC promises to end the secrecy, but there's a lot of potential gain in being able to simplify bundles to min-set for author maintenance and generate the full data only later (incl channel). Right now the vernacular for this is a 'veneer' (should be something more like a 'shim', since a 'veneer' is a thin something applied to make something look like another, whereas a 'shim' is something thin which exists between two others; a slim transitional step).

## Terms
FBC := File-based Catalogs
OLM := Operator Lifecycle Manager; framework for k8s-native applications called 'operators'
OPM := package manager for 'bundles' which define 'operators' at a version as well as upgrade paths for the operator
channel := one view of a graph for upgrades; analogous to the RPM database yum uses for dependency resolution

## Impetii:
- 'opm add' doesn't work for operator author (broken database).... usability, least-surprise philosophy
- cluster-admin operator: cluster-admin reqs being seamlessly provided by operator-authors, consistency
- 4 official catalogs released, plus custom catalogs (e.g. from IBM)
- stealing dev resources to build the catalogs in haste to support a release (and old sql out of expertise scope for much of the team)

### The answer: catalog templates ###
Initial work
- https://hackmd.io/J6PQYDnLTfqYGLWpOs44aQ?view
- https://hackmd.io/ixIuRBNURV-7K4C1aGkzxQ
- https://github.com/joelanford/example-operator-index

## IN THE BEGINNING ...
- package-manifest, with custom commands for quay for application-repository access
- revamped in favor of catalog images
- bundles analogous to RPMs, containing app
- catalog like index of repository, including versions, etc.

### opm tool ###
- imperative access to add bundle to/remove bundle from index
- supply index, bundle, it would figure out how to add bundle to index
- upgrade graph, APIs include upgrade edges for consideration
- replaces
- skips
- skip-range
- deprecate/truncate removes tail of upgrade graph

same bundles can (likely will) exist in multiple channels

*FBC was to un-hide all the info in the databases, making input data transparent to make it possible to manipulate bundle/channel without using opm tool (though opm also enhanced to interpret/convert-to FBCs)*

## WHERE WE ARE NOW
- the lowest level of APIs fed into a catalog image, the server knows how to serve in the cluster (low-level, public)
- v4.11 "boot-strapped", first FBC catalogs to be shipped
- **phase 1**, hides that we still use a DB behind the scenes, pretends that we are all plaintext
(4 diff catalogs, 3 diff piplines using 'opm registry add' under the hood, re-generates the image downstream as FBC)
- customer can only affect the catalog on a per-bundle basis, so we have some edge-cases to recognize when metadata shifts as the result of a bundle add operation
- **phase 2**, enable folks to add as bundle + catalog, and we no longer build the catalogs from intuited/interpreted data of the bundle
- argument is to deploy a single catalog template type in the downstream (RH-specific) stream first
- first step, after lay-of-the-land, is to build a catalog template (arbitrary input with algo that produces FBC): semver-based first
- take a given bundle image off-the-shelf, and generate the template from a bundle we can demonstrate can compose a channel
- policy-based development
- don't make API changes to the cluster, in service to the number of operator authors currently utilizing the space
- (deppy, rukpak) := newer APIs can interact with evolving FBC APIs
- **phase 2 demo**: simplest demo of ingest bundle ... emit catalog in repo (NB: not deployment, just conversion of template to full catalog so it could be committed & deployed later)

## References
### APIs
- OLM.package
- OLM.channel
- OLM.bundle

### next steps ###
OLM-2455:
- build bundle
- demonstrate catalog-of-catalogs generation from veneer on bundle build
- stretch: automate the building
- stretch-stretch: do it in go
- promote bundle in a channel

[next > demo design](https://hackmd.io/EIWEwm28TZiwHUG25cmLBw)