owned this note
owned this note
Published
Linked with GitHub
# e2e strawman v0.6.2
OLM would be replaced by three projects:
- RukPak, which deals with bundles and their application on a cluster
- Deppy, which deals with indexes and resolving content from them
- Operator Provisioner, which plugs into Depster/RukPak to give them support for OLM-operator bundles
The division between these components arose naturally when planning the concerns for the new APIs - giving them different names / subprojects highlights their independence (but may not be reflect the short-term implementation plan).
Each component builds on the other and is explained in detail below.
# RukPak
RukPak is a pluggable solution for the packaging and distribution of cloud-native content and supports advanced strategies for installation, updates, and policy.
RukPak provides a content ecosystem for clusters:
- `Bundle`s contain remote or local content. This includes remote image bundles in the manifest bundle format, local volumes, git repos, etc.
- `Instance`s indicate that the content of a `Bundle` should be active in a cluster - think `Pod` but for arbitrary artifact types.
Different types of `Bundles` can be supported by adding `provisioners` to a cluster, much in the same way as persitstent volume types or CSI drivers. The unpacking and installation of bundles can be configured via `ProvisionerClass`es at runtime.
APIs and Components:
```mermaid
graph LR
subgraph RukPak APIs
Class(ProvisionerClass)
Bundle(Bundle) -.-> |has a| Class
Instance(Instance) -.-> |has a| Class
end
subgraph Running Components
Provisioners[Provisioners<br>Operator,Helm,etc] --> Instance & Bundle & Class
end
classDef future fill:#ddd;
```
## Bundle API
A`Bundle` represents content that needs to be made available to other consumers in the cluster.
Much like the contents of a container `image` need to be pulled and unpacked in order for `Pod`s to start using them, `Bundle`s are used to reference content that may need to be pulled and should be unpacked. In this sense, `Bundle` is a generalization of the`image` concept.
The specifics of how a `Bundle` get unpacked and consumed are defined by the `ProvisionerClass` and the `provisioner` that is configured to handle that `ProvisionerClass`.
```yaml
kind: Bundle
metadata:
name: plumbus-operator.v0.9.3
spec:
class: deppy.resolveset
refs:
- file://content
volumeMounts:
- mountPath: /content
configMap:
name: local
namespace: plumbus
```
`Bundles` do nothing on their own - they require a provisioner to unpack and make their content available.
## Instance API
The `Instance` API points to a `Bundle` and indicates that it should be "active". This includes pivoting from older versions of a active bundles. `Instance` may also include an embedded spec for a desired `Bundle`.
Much like `Pod`s stamp out instances of `images`, and `Instance` stamps out an instances of `Bundles`.`Instance` can be seen as a generaliztion of the `Pod` concept.
The specifics of how an `Instance` makes changes to a cluster based on a referenced `Bundle` is defined by the `ProvisionerClass` and the `provisioner` that is configured to handle that `ProvisionerClass`.
```yaml
kind: Instance
metadata:
name: resolved-654adh
spec:
selector:
matchLabels:
subscription: etcd-operator
bundle:
name: resolved-654adh
spec:
class: olm.resolveset
refs:
- file://content
volumeMounts:
- mountPath: /content
configMap:
name: resolved-654adh-content
namespace: olm
```
## ProvisionerClass API
ProvisionerClass defines a configuration for a provisioner. Provisioners are `controllers` that understand `Instance`, `Bundle`, and `ProvisionerClass` APIs and take action.
Each `provisioner` has a unique id. For example, the provisioner that ships with OLM and understand operator bundles has the id: `operators.coreos.com/operators`.
A `ProvisionerClass` specifies a specific configuration of provisioning (i.e. settings to interpret `Instance` and `Bundle` apis). A `provisioner` will only operate on `Instance` and `Bundle` objects that reference a `ProvisionerClass` that contain the `provisioner`'s unique id.
If this seems familiar, it is the same pattern that is used by `StorageClass` / `PeristentVolume` / `PersistentVolumeClaim` in kubernetes.
Example `ProvisionerClass`:
```yaml
kind: ProvisionerClass
apiVersion: rukpak.io/v1
metadata:
name: <good name>
provisioner: <unique id>
# parameters has no schema, is provisioner-specific
parameters: {}
```
### ProvisionerClass Permission
`ProvisionerClass` comes with a `use` verb.
Only users and serviceaccounts with `use` on a `ProvisionerClass` can create an object that references that `ProvisionerClass`.
A webhook rejects requests on `Instance`, and `Bundle` that reference `ProvisionerClass`es that the requesting user/serviceaccount does not have the `use` verb on.
## Approval
Approval is an implementation detail of provisioners, which may choose to block rollout until certain conditions are met.
Instances have an `Approved` condition:
```yaml
kind: Instance
metadata:
name: resolved-654adh
spec:
# omitted
status:
conditions:
- lastUpdateTime: "2020-02-08T11:37:35Z"
lastTransitionTime: "2020-02-08T11:37:35Z"
message: Approved by my custom approver controller
reason: ApprovedByMyPolicy
status: True
type: Approved
```
A webhook ships with RukPak that prevents any update to the `Approved` condition by any user or service account that lacks the `approve` verb on the `Instance` API.
The provisioner for the specified provisioner class is responsible for determining if approval is required or not. The provisioner may choose to auto-approve by writing the `Approved` condition into the status, or it may choose to leave the `Instance` unapproved. A user, or a separate approval plugin, may take over the approval from there.
`kubectl instance approve <my-instance>` will approve an instance that is awaitng approval, as long as the user has sufficient permission.
---
---
---
# Deppy
Deppy runs on- or off-cluster to provide dependency resolution for catalogs of RukPak bundles.
Deppy provides an API for resolving constraints over catalog content (see: [declarative index format](https://github.com/operator-framework/enhancements/blob/master/enhancements/declarative-index-config.md)).
The on-cluster installation of Deppy also includes:
- Continual (optional) resolution against catalog content, to keep installed packages up-to-date.
- A RukPak `provisioner` and default `ProvisionerClass` for `deppy.resolveset` bundles.
- An API for defining constraints at runtime (`Input`)
- A RukPak Approval plugin that provides Android-style update policy
## ResolveSet Bundles
This is an example of a `Bundle` for a local `deppy.resolveset` bundle. It declares the the contents specified by `refs` are needed locally by other consumers in the cluster.
The controller watching this `Bundle` decides how to unpack and is configured via `spec.class`. In this case, the provisioner knows that the `resolveset` lists out other `Bundles` that should be created on the cluster, and begins to create them.
```yaml
kind: Bundle
metadata:
name: etcd-operator.v0.9.3
spec:
class: deppy.resolveset
refs:
- file://content
volumeMounts:
- mountPath: /content
configMap:
name: resolved-654adh-content
namespace: olm
status:
unpacked: NotStarted | InProgress | Done
objects:
- /objects/resolved.bundles.json
---
apiVersion: v1
kind: ConfigMap
metadata:
name: resolved-654adh-content
namespace: default
data:
resolved.bundles.json: |-
{
"schema": "bundle.v1",
"packageName": "quay.io/operatorhubio/etcd",
"path": "quay.io/operatorhubio/etcd:v0.6.1",
"version": "0.6.1",
"properties": [
{
"name": "pivotFrom",
"value": "etcd-v0.6.0"
},
{
"name": "olm.gvk",
"value": {
"group": "etcd.database.coreos.com",
"version": "v1beta2",
"kind": "EtcdCluster"
}
},
],
"channels": [
"alpha"
],
},
{
"schema": "bundle.v1",
"packageName": "quay.io/operatorhubio/prometheus",
"version": "1.0.0",
"properties": [
{
"name": "olm.label",
"value": "LTS",
}
]
}
```
## ResolveSet Provisioner Configuration
Deppy ships with one provisioner, `deppy.io/resolver`, that understands `resolveset` bundles and knows how to unpack, apply, and pivot them.
The default `ProvisionerClass`es configure one of these provisioners to support common `Instance` and `Bundle` workflows.
```yaml
kind: ProvisionerClass
apiVersion: rukpak.io/v1
metadata:
name: olm.resolveset
provisioner: operators.coreos.com/olm
parameters:
approval: AllowAll | DenyAll | Android
# for a given bundle entry, determine what ProvisionerClass to use when stamping out a Bundle
matchBundle:
- selector: ".schema == olm.bundle"
class: olm.bundle
- selector: ".schema == helm.chart"
class: helm.bundle
# equivalent to above when picking ProvisionerClass based on schema
matchSchema:
- schema: olm.bundle
class: olm.bundle
```
### RukPak Approval Plugin
The resolveset provisioner understands the folliowing parameter in `ProvisionerClass`
```yaml
kind: ProvisionerClass
parameters:
approval: AllowAll | DenyAll | Android
```
- `AllowAll`: Any Instance for a Bundle using a ProvisionerClass with this setting will be created with the `Approved` condition.
- `DenyAll`: Any Instance for a Bundle using a ProvisionerClass with this parameter will be created without the `Approved` condition. Another actor (human, automation) can add `Approved` after the fact.
- `Android`: Any Instance for a Bundle using a ProvisionerClass with this parameter will be created with the `Approved` condition if it can be determined that no bundle exceeds the permission of the previously installed bundle.
The value of `approval` for the default `deppy.resolveset` `ProvisionerClass` is `Android`.
## Input API
The `Input` APIs are used to create inputs for the resolver. An instance of `Input` is more or less an `Installable` (in internal resolver parlance).
```yaml
apiVersion: deppy.io/v1
kind: Input
metadata:
name: plumbus
spec:
inputClass: subscription
constraints:
# most common - equivalent to a Subscription
- type: olm.catalog
value:
name: community
namespace: my-ns
package: plumbus
channel: stable
# other examples of constraints - less common, but useful
# lock to a specific version by package/version
- type: olm.packageVersion
value:
package: "plumbus"
version: v2.0.0
# lock to a specific version by name
- type: olm.name
value: "plumbus.v2.0.0"
# restrict to a range of versions
- type: olm.packageVersion
value:
package: "plumbus"
version: ">2.0.0 <3.0.0"
# arbitrary rego queries
- type: olm.rego
value: "semver.Compare(minOCPVersion, 4.8.0) == 1"
status:
# instances with properties that match the constraints
instances:
- name: plumbus.v2.0.0-alpha
# other instances satisfied by this instance
satisfies:
- kind: Instance
name: other-operator
apiVersion: rukpak.io/v1
meets:
- {"olm.constraint": {"olm.name": "etcd-operator.v0.9.3"}}
# dependencies introduced by this instance
dependencies:
- kind: Instance
name: prometheus-operator-abc
apiVersion: rukpak.io/v1
meets:
- {"olm.constraint": {"olm.gvk": {"group": "manufacturing.how.theydoit.com", "version": "v1alpha1", "kind": "Grumbo"}}
- {"olm.constraint": {"olm.label": "LTS"}}
# isntance-specific conditions
conditions:
- type: DependenciesMissing
status: True
reason: Only 2/3 dependency constraints are met.
message: "etcd-operator has the following constraints: X,Y,Z. X and Z are satisfied by prometheus-operator-abc, but no instance satisfies Y"
lastTransitionTime: "2019-09-16T22:26:29Z"
# conditions about the input itself
conditions:
- type: ResolutionFailed
message: "unable to find a solution that matched constraints: X requires Y, but Z requires !Y, X and Z are mandatory"
status: True
```
## InputClass API
InputClass defines a configuration for an `Input`.
End-users may select, but will likely never write, an `InputClass`.
### Default InputClasses
Deppy ships with a set of default InputClasses
#### Simulate `Subscription` behavior
```yaml
kind: InputClass
apiVersion: deppy.io/v1
metadata:
name: subscription
parameters:
# if force = true, all matching entries will be installed
# directly, and the resolver will not be engaged
force: false
# if true, new `Inputs` will be generated for any Instances
# that have been installed as a dependency
generateInputsForDependencies: true
# if true, does not output an `Instance`
dryRun: false
# if true, the resolveset bundle includes a lot of data about the resolution process
trace: false
# any constraints defined here are included automatically in the `Input` constraints that reference this class
constraints:
- type: minKubeVersion
value: >= 1.20
- type: minOCPVersion
value: >= 4.7
```
#### Force install operators
```yaml
kind: InputClass
apiVersion: deppy.io/v1
metadata:
name: force
parameters:
force: true
generateInputsForDependencies: false
dryRun: false
trace: false
constraints:
- type: minKubeVersion
value: >= 1.20
- type: minOCPVersion
value: >= 4.7
```
#### Install dependencies, but don't keep them up-to-date independently (i.e. minimal version selection)
```yaml
kind: InputClass
apiVersion: deppy.io/v1
metadata:
name: minimal-version
parameters:
force: true
generateInputsForDependencies: false
dryRun: false
trace: false
constraints:
- type: minKubeVersion
value: >= 1.20
- type: minOCPVersion
value: >= 4.7
```
#### Resolve but don't apply
```yaml
kind: InputClass
apiVersion: deppy.io/v1
metadata:
name: dry-run
parameters:
force: false
generateInputsForDependencies: false
dryRun: true
trace: false
constraints:
- type: minKubeVersion
value: >= 1.20
- type: minOCPVersion
value: >= 4.7
```
```yaml
kind: InputClass
apiVersion: deppy.io/v1
metadata:
name: dry-run-next
parameters:
force: false
generateInputsForDependencies: false
dryRun: true
trace: false
# set to the next minor release of kube/ocp
constraints:
- type: minKubeVersion
value: >= 1.21
- type: minOCPVersion
value: >= 4.8
```
## CatalogSource API
Just the normal CatalogSource API. Cluster scoped?
## Resolver API
Note: See Future Work notes at the end of the doc for caveats / considerations.
The resolver has an interactive (**non-kube**) API that supports:
### Queries
Input:
- a constraint
- (or) a rego query
Output:
- a list of matching bundles from the catalog cache
### Interactive Resolution
Input:
- a list of lists of constraints (i.e. a list of Inputs)
Output:
- resolved set, or error
---
---
---
# Operator Provisioner
The Operator provisioner understands `registry+v1` manifest bundles and instances, providing support for `olm.bundle` entries in catalogs. It also comes with:
- Default ProvisionerClasses for operator bundles
- An approval plugin for operator bundles
- A set of default `InputClass`es for use when installing operator bundles
- A porcelein `Operator` API that can be used for installing operator bundles in lieu of writing `Input` directly.
### Operator Bundle Example
This is an example of a `Bundle` for a remote `registry+v1` bundle image. It declares the the contents specified by `refs` are needed locally by other consumers in the cluster.
The controller watching this `Bundle` decides how to unpack and is configured via `spec.class`. In this case, the provisioner knows that this is `registry+v1` bundle, and unpacks the `manifests` held within.
The `status` is controlled by the `provisioner` configured on the `class`, but it lists out locations in the cluster that unpacked content can be found.
```yaml
kind: Bundle
metadata:
name: etcd-operator.v0.9.3
spec:
class: olm.bundle
refs:
- docker://quay.io/etcd/bundle@sha256
status:
unpacked: NotStarted | InProgress | Done
objects:
- /objects/csv
- /objects/role
```
### Operator Bundle Instance Example
This is an example of an `Instance` that declares that `registry+v1` bundle image should be pulled down and installed on the cluster.
The `selector` may match other (older) bundles. The controller watching this instance decides how to pivot and is configured via `spec.bundle.spec.class`, or on `spec.class` of the `Bundle` object that `Instance` points to.
```yaml
kind: Instance
spec:
# selector is used to find previous/current Bundles
selector:
matchLabels:
subscription: etcd-operator
# a reference to the Bundle object that should be considered "Active".
bundle:
name: etcd-operator.v0.9.3
# optional, for direct application of a bundle definition
spec:
class: olm.bundle
refs:
- docker://quay.io/etcd/bundle@sha256
# or: volume-backed
status:
conditions:
- type: Approved
status: True
reason: ApprovedByMyPolicy
message: Approved by my custom approver controller
lastTransitionTime: "2020-02-08T11:37:35Z"
```
### Default ProvisionerClasses
The default `ProvisionerClass`es configure one of these provisioners to support common `Instance` and `Bundle` workflows.
```yaml
kind: ProvisionerClass
apiVersion: rukpak.io/v1
metadata:
name: olm.bundle
provisioner: operators.coreos.com/operators
parameters:
approval: AllowAll | DenyAll | UserPermission
# options for unpacking bundles
maxObjectSize: 10mb
```
#### Approval for the operator provisioner
The operator provisioner understands the folliowing parameter in `ProvisionerClass`
```yaml
kind: ProvisionerClass
parameters:
approval: AllowAll | DenyAll | UserPermission
```
- `AllowAll`: Any Instance for a Bundle using a ProvisionerClass with this setting will be created with the `Approved` condition.
- `DenyAll`: Any Instance for a Bundle using a ProvisionerClass with this parameter will be created without the `Approved` condition. Another actor (human, automation) can add `Approved` after the fact.
- `UserPermission`: Any Instance for a Bundle using a ProvisionerClass with this parameter will be created with the `Approved` condition if it can be determined via `SubjectAccessReview` checks that the user that created it has sufficient permission to create all of the contents of the Bundle.
The value of `approval` for the default `olm.bundle` `ProvisionerClass` is `UserPermission`.
### Operator API
The simplest way to install operators and subscribe to updates is via the `Operator` API. The current status and install progress, as well as references to all components of an operator, can be found on the `status` block of the `Operator` object.
Example:
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: subscription
# optional
# this field is translated directly to an `Input` constraint
# that looks like:
#
# type: olm.catalog
# value:
# name: community
# namespace: my-ns
# package: plumbus
# channel: stable
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
# optional, lock to a specific version
# this field is translated directly to an `Input` constraint
# that looks like:
#
# type: olm.packageVersion
# value:
# package: plumbus
# version: v2.0.0
#
version: "2.0.0"
# version may also be a semver range, like ">2.0.0 <3.0.0"
version: ">2.0.0 <3.0.0"
# optional
# additional arbitrary constraints can be added
constraints:
- type: olm.rego
value: "semver.Compare(minOCPVersion, 4.8.0) == 1"
status:
updates:
- options:
- plumbus.v2.0.8
- plumbus.v2.0.7
- plumbus.v2.0.6
catalogSourceRef:
name: community
namespace: my-ns
package: plumbus
channel: beta
conditions:
- kind: UpdateAvailable
status: True
reason: CrossChannelUpdateFound
message: updates have been found that require a change in the operator spec to apply
lastTransitionTime: "2019-09-16T22:26:29Z"
components:
matchLabels:
operators.coreos.com/plumbus: ""
refs:
- kind: Input
name: plumbus
apiVersion: rukpak.io/v1
- kind: Instance
name: plumbus.v2.0.0-alpha
apiVersion: rukpak.io/v1
conditions:
- type: Installing
status: true
- kind: Bundle
name: plumbus.v2.0.0-alpha
apiVersion: rukpak.io/v1
conditions:
- type: Unpacked
status: true
# older bundles visible during pivot
- kind: Bundle
name: plumbus.1.9.0
apiVersion: rukpak.io/v1
conditions:
- type: Unpacked
status: true
- type: Replaced
status: true
- kind: ClusterServiceVersion
namespace: operators
name: plumbus.v2.0.0-alpha
apiVersion: rukpak.io/v1alpha1
conditions:
- type: Installing
status: True
reason: AllPreconditionsMet
message: deployment rolling out
lastTransitionTime: "2019-09-16T22:26:29Z"
- kind: CustomResourceDefinition
name: plumbai.how.dotheydoit.com
apiVersion: apiextensions.k8s.io/v1beta1
- kind: ClusterRoleBinding
namespace: operators
name: rb-9oacj
apiVersion: rbac.authorization.k8s.io/v1
```
When an `Operator` is created, an appropriate `Input` will be created. The `Operator` controller makes use of the Resolver API to determine a set of potential updates to highlight in the status (i.e. by querying in other channels / relaxing constraints).
Most users will not need to look at other APIs.
# Future Work: Constraints, MetaConstraints, and Preference
This topic has not been nailed down to a degree that a strawman made sense. Instead, the scope of the problem is outlined below.
There is a big limitation in the current model (in `deppy`, specifically), which limits the constraints and meta-constraints that can be specified as input.
For example, this constraint:
```yaml
type: olm.catalog
value:
name: community
namespace: my-ns
package: plumbus
channel: stable
```
Gets translated to a boolean formula for the sat-solver that looks like:
```
[A or B or C or D]
```
Where `A`, `B`, `C`, and `D` are all of the operators in the `stable` channel for the `plumbus` package. This constrains all solutions to only those that can be found in the specific package/channel, encoding it in a way that the solver understands.
Today OLM does the translation of constraints to these boolean formulas (we call a single formula an "installable" in the OLM codebase).
The resolver also has a set of constraints that enforce properties and invariants about the solution. Today these are things like "only one subscription to a package can exist in a channel" or "only one operator can provide a gvk in a namespace".
These "meta-constraints" are also translated into boolean formulas and fed to the solver.
In the model above, each new constraint type or constraint needs to be encoded in `deppy`, because `deppy` does the translation of constraints to solver inputs (boolean formulas).
We know that, when building a package ecosystem, it is useful to be able to define this type of constraint. We would like to be resilient to future extension needs (i.e. it should be straightforward to add helm-specific constraint types, as the need arises).
The `rego` constraint type gives us some flexiblity, but limits us to filtering that can be expressed as a rego query.
Some options we have discussed:
- Constraints should all be specified as rego queries
- Define plugin system that can be called directly from deppy (wasm?) for filtering
- Define an InputClass "provisioner" that provides an API that converts from a constraint to a boolean formula
- A `Solve` API that takes as input boolean constraints. Plugins translate constraints on `Input` into boolean constraints on `Solve`, the resolver outputs its results into the status of `Solve.`
Preference is also not currently pluggable.
Preference is the property in the resolver that tests solution in a particular order - it's the reason that an operator from the head of a channel is picked instead of an older release, if it can be. It's the reason operators from the redhat-catalog are preferred to operators in the community-catalog, and why dependencies are preferred from the same catalog from which they came.
As we expand to other artifact types, it's clear that we would like to be able to define preference order for other constraints.
# Changelog
- 0.6.2
- migrated doc ownership
- 0.6.1
- move satisfies/dependency info to Inputs
- add back in user stories
- renamed depster -> deppy
- 0.6
- cleanup
- clarify pluggability questions / future work
- 0.5
- One project for instance / bundle (rukpak)
- One project for resolver + resolver apis (depster)
- Plugins for those
- 0.4
- Separate operator-specific things from resolver+bundle things
- 0.3
- Added an arch diagram
- Named resolver+bundle subproject `RukPak` with new group `rukpak.io`
- `Operator` API puppets `Input` API for now
- Error messages on `Input` API
- Adds option for tracing resolver output, added to resolveset
- Add sketch of Resolver API to answer questions about available updates / dry-run
- Added a note about future extension via `InputMetaClass`
- 0.2
- add an FAQ
- add a new API (`Input`) for constraints, separate from Operator
- re-usable across other artifact types
- factor out the top-level fields in Operator
- entrypoint when installing somethign that is not an Operator
- make the `Approved` condition match other condition apis
- 0.1
- initial e2e: do we get everything we need with just Operator + Bundle + Instance?
---
---
---
# User Stories
Evaluate the design against target user stories.
### Force install a single operator from a catalog: don't subscribe it to updates, don't resolve dependencies
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: force
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
version: "2.0.0"
```
- Creates an `Input` with matching constraints
- Creates an `olm.resolveset` Instance that contains every bundle that matches the constraints
- This may pick multiple bundles
- This does not fulfil dependencies - the resolveset contains only the bundles that match the query
### Install an operator and its dependencies from a catalog but don't subscribe it to updates.
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: minimal-version
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
version: "2.0.0"
```
- Creates an `Input` with matching constraints
- Creates a `resolveset` Instance
- Creates a set of Instances for all bundles
- Constraints lock this to a particular version, no updates occur
### Install a single operator from a catalog and subscribe it to updates, but don't subscribe its dependencies to updates.
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: minimal-version
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
```
- Creates an `Input` with matching constraints
- Creates a `resolveset` Instance
- creates a set of Instances for all bundles
- Constraints allow for updates
### Install an operator and its dependencies and subscribe all of the operator and all of its dependencies to updates.
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: subscription
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
```
- Creates an `Input` with matching constraints
- Creates a `resolveset` Instance
- creates a set of Instances for all bundles
- For each Instance in the `resolveset`, creates an `Input` with appropriate constraints
### Install a single operator bundle directly (no catalog)
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
bundle:
name: etcd-operator.v0.9.3
spec:
class: olm.bundle
refs:
- docker://quay.io/etcd/bundle@sha256
```
which results in:
```yaml
kind: Instance
spec:
# selector is used to find previous/current Bundles
selector:
matchLabels:
subscription: etcd-operator
# a reference to the Bundle object that should be considered "Active".
bundle:
name: etcd-operator.v0.9.3
# optional, for direct application of a bundle definition
spec:
class: olm.bundle
refs:
- docker://quay.io/etcd/bundle@sha256
# or: volume-backed
status:
# progress, failure, etc
#
```
### Update the dependencies of an operator even if there is no "subscription" for the dependencies.
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: minimal-version
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
```
- Creates an `Input` with matching constraints
- `minimal-version` means that the dependencies will only update when this one has requirements on newer versions.
### Force update a single operator directly to a specific version, with no catalog available in the cluster.
Assuming there is an existing `Instance` on the cluster for `etcd-operator`:
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
bundle:
name: etcd-operator
spec:
class: olm.bundle
refs:
- docker://quay.io/etcd/bundle@sha256
```
will result in:
```yaml
kind: Instance
spec:
# selector that matches the existing Bundle we're updating
selector:
matchLabels:
subscription: etcd-operator
# a reference to the Bundle object that should now be considered "Active".
bundle:
name: etcd-operator
spec:
class: olm.bundle
# ref to the bundle we want to update to
refs:
- docker://quay.io/etcd/bundle@sha256
status:
# progress, failure, etc
#
```
- This attempts to directly update an installed bundle to another bundle, pivoting ownerrefs, etc
### Force update a single operator directly to a specific version from a catalog
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: force
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
version: "2.0.0"
```
- Creates an `Input` with matching constraints
- Creates an `olm.resolveset` Instance for every bundle that matches the constraints, with appropriate "pivot" data added.
### Able to change the properties or constraints of any installed bundle.
Properties and constraints, as determined by the resolver, are persisted as annotations on the Bundle. They can be edited by a user to change the way resolver sees them (this is generally inadvisable and should only be done when absolutely necessary).
```yaml
kind: Bundle
apiVersion: rukpak.io/v1
metadata:
name: plumbus-kjf122
annotations:
olm.properties: -|
{"olm.gvk": {"group": "how.theydoit.com", "version": "v2alpha1", "kind": "Plumbus"}}
{"olm.constraint": {"olm.gvk": {"group": "manufacturing.how.theydoit.com", "version": "v1alpha1", "kind": "Grumbo"}}
spec:
class: registry.v1
refs:
- docker://quay.io/plumbus/bundle@sha256:abcdef12345
status:
unpacked: Done
objects:
- /objects/csv
- /objects/role
```
### Communicate to the user when the things that are already installed have unsatisfied dependencies and communicate to a user that dependencies are satisfied by specific other bundles.
Status is provided on Input. This is aggregated to the `Operator` status via existing mechanisms.
```yaml
apiVersion: depster.io/v1
kind: Input
metadata:
name: plumbus
spec:
inputClass: subscription
constraints:
- type: olm.catalog
value:
name: community
namespace: my-ns
package: plumbus
channel: stable
status:
# instances with properties that match the constraints
instances:
- name: plumbus.v2.0.0-alpha
# other instances satisfied by this instance
satisfies:
- kind: Instance
name: other-operator
apiVersion: rukpak.io/v1
meets:
- {"olm.constraint": {"olm.name": "etcd-operator.v0.9.3"}}
# dependencies introduced by this instance
dependencies:
- kind: Instance
name: prometheus-operator-abc
apiVersion: rukpak.io/v1
meets:
- {"olm.constraint": {"olm.gvk": {"group": "manufacturing.how.theydoit.com", "version": "v1alpha1", "kind": "Grumbo"}}
- {"olm.constraint": {"olm.label": "LTS"}}
# isntance-specific conditions
conditions:
- type: DependenciesMissing
status: True
reason: Only 2/3 dependency constraints are met.
message: "etcd-operator has the following constraints: X,Y,Z. X and Z are satisfied by prometheus-operator-abc, but no instance satisfies Y"
lastTransitionTime: "2019-09-16T22:26:29Z"
```
### Communicate to the users when they are about to install operators, which dependencies would be resolved
- Installing new content by default will result in an Instance waiting to be approved (due to android permissions)
- User reviews the content of the instance before `kubectl instance approve`
- Users that want more control can use a provisionerClass with `DenyAll` so that every single change requires review.
### Don't block updates to operators when other unrelated operators are blocked.
The resolver will generate disjoint resolveset `Instance`s.
### Understand why an update is blocked
The resolver will emit events in the namespace.
Most other states should be readily visible in current `Input` (and `Operator`) status
### If an operator is not subscribed to updates, be presented with options for update as they become available (OCP-style).
- If using the `Operator` API, the status will include information about potential updates
- The resolver API can be queried with constraints in the general case.
### Operator updates available in other catalogs or channels are surfaced to users.
- If using the `Operator` API, the status will include information about potential updates
- The resolver API can be queried with constraints in the general case.
### Ability to write policy for updates. Andriod-style permission policy comes by default.
Instances may have an `Approved` condition:
```yaml
kind: Instance
metadata:
name: resolved-654adh
spec:
selector:
matchLabels:
subscription: etcd-operator
bundle:
name: resolved-654adh
spec:
class: olm.resolveset
refs:
- file://content
volumeMounts:
- mountPath: /content
configMap:
name: resolved-654adh-content
namespace: olm
status:
conditions:
- lastUpdateTime: "2020-02-08T11:37:35Z"
lastTransitionTime: "2020-02-08T11:37:35Z"
message: Approved by my custom approver controller
reason: ApprovedByMyPolicy
status: True
type: Approved
```
Depster ships with approval plugins as part of the `olm.resolveset` provisioner, the operator provisioner ships with approval plugins for `olm.bundle`. Additional plugins can be written, configured, and installed as needed.
### Non-administrators should be able to install non-operator content, as long as they have sufficient permission for the install.
Strawman: As a non-administrator user, I would like to trigger a namespace-local installation of a helm chart via a bundle.
A helm provisioner is installed that watches `Instance`, and `Bundle`. It may also choose to include porcelein APIs that translate to `Inputs`
When I create an `Instance` that contains references a provisionerclass that points to the helm provisioner, the helm provisioner is used to install.
The helm provision may choose to provide additional apis for visibility or control (just as olm provides the `Operator` API).
The helm provisioner may provide a `HelmChart` API that is namespaced-scoped and performs access checks for a user before creating an `Input` on a user's behalf.
Strawman 2: An additional namespace-scoped API can be added, `OperatorRequest`, that either results in the creation of an unapproved `Instance`
### As a non-administrator user, I would like to request an Operator install in the style of Kubernetes certificate signing requests getting approved by admins
This should more or less be covered by the approval workflow + a way to request installation with namespace scoped APIs.
### Determine what would be installed under specific conditions - i.e. if the cluster were updated to a new version, if a specific operator is selected
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: dry-run
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
version: "2.0.0"
```
results in
```yaml=
apiVersion: rukpak.io/v1
kind: Input
metadata:
name: plumbus
spec:
inputClass: dry-run
constraints:
- type: olm.catalog
value:
name: community
namespace: my-ns
package: plumbus
channel: stable
- type: olm.packageVersion
value:
packageName: plumbus
version: "2.0.0"
```
The `dry-run` inputClass sets `dryRun: true`. When dry-run is true, the resolver will output a resolveset `Bundle` but not a resolveset `Instance`, so no changes occur to running bundles.
Alternatively, the resolver API can be queried interactively for this information.
### If a dependency is in a degraded state, I can tell this from the top-level operator.
The `Operator` aggregates status of its components, which includes `Input` objects, which report dependency (satisfaction) problems.
Actual dependency health: TODO
### Install at a particular version and then enable updates.
1. Create an `Operator` to install at a specific version
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: minimal-version
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
version: "2.0.0"
```
2. Edit the `Operator` to enable updates
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: minimal-version
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
```
### Install at a particular version, and walk through a set of updates, stopping before hitting latest.
1. Create an `Input` to install at a specific version
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: minimal-version
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
version: "2.0.0"
```
2. Edit the `Input` to enable updates up to a specific release
```yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
name: plumbus
spec:
inputClass: minimal-version
catalog:
name: community
namespace: my-ns
package: plumbus
channel: stable
version: "<= 2.4.5"
```
## Install two operators that own the same APIs
## Adjacent stories
Stories that are best addressed with features outside the scope of this enhancement.
### View the update graph for a particular operator in the console.
The package server can be queried for this data, the console can display it.
### Present a user with release notes for available updates, and with release notes for currently installed operators.
Given any `bundle`, the package-server can be queried for release notes.
Detailed release notes or docs may be part of an operator's bundle, visible after install.
### Non-administrators can get info about operator-provided APIs based on their access to them.
Options:
- Hack into discovery / openapi
- Have a cluster-scoped api just for this info
- Add extended property support to kube
- "man pages" for kube