# e2e strawman v0.6.2

OLM would be replaced by three projects:

- RukPak, which deals with bundles and their application on a cluster
- Deppy, which deals with indexes and resolving content from them
- Operator Provisioner, which plugs into Depster/RukPak to give them support for OLM-operator bundles

The division between these components arose naturally when planning the concerns for the new APIs - giving them different names / subprojects highlights their independence (but may not be reflect the short-term implementation plan).

Each component builds on the other and is explained in detail below.

# RukPak

RukPak is a pluggable solution for the packaging and distribution of cloud-native content and supports advanced strategies for installation, updates, and policy.

RukPak provides a content ecosystem for clusters:

- `Bundle`s contain remote or local content. This includes remote image bundles in the manifest bundle format, local volumes, git repos, etc.
- `Instance`s indicate that the content of a `Bundle` should be active in a cluster - think `Pod` but for arbitrary artifact types.

Different types of `Bundles` can be supported by adding `provisioners` to a cluster, much in the same way as persitstent volume types or CSI drivers. The unpacking and installation of bundles can be configured via `ProvisionerClass`es at runtime.

APIs and Components:

graph LR
subgraph RukPak APIs
     Class(ProvisionerClass)
     Bundle(Bundle) -.-> |has a| Class
     Instance(Instance) -.-> |has a| Class
end

subgraph Running Components
Provisioners[Provisioners<br>Operator,Helm,etc] --> Instance & Bundle & Class
end


classDef future fill:#ddd;

## Bundle API

A`Bundle` represents content that needs to be made available to other consumers in the cluster.

Much like the contents of a container `image` need to be pulled and unpacked in order for `Pod`s to start using them, `Bundle`s are used to reference content that may need to be pulled and should be unpacked. In this sense, `Bundle` is a generalization of the`image` concept.

The specifics of how a `Bundle` get unpacked and consumed are defined by the `ProvisionerClass` and the `provisioner` that is configured to handle that `ProvisionerClass`.

kind: Bundle
metadata: 
  name: plumbus-operator.v0.9.3
spec:
  class: deppy.resolveset
  refs:
  - file://content
  volumeMounts:
  - mountPath: /content
    configMap:
      name: local 
      namespace: plumbus

`Bundles` do nothing on their own - they require a provisioner to unpack and make their content available.

## Instance API

The `Instance` API points to a `Bundle` and indicates that it should be "active". This includes pivoting from older versions of a active bundles. `Instance` may also include an embedded spec for a desired `Bundle`.

Much like `Pod`s stamp out instances of `images`, and `Instance` stamps out an instances of `Bundles`.`Instance` can be seen as a generaliztion of the `Pod` concept.

The specifics of how an `Instance` makes changes to a cluster based on a referenced `Bundle` is defined by the `ProvisionerClass` and the `provisioner` that is configured to handle that `ProvisionerClass`.

kind: Instance
metadata:
  name: resolved-654adh
spec:
  selector: 
    matchLabels:
      subscription: etcd-operator
  bundle:
    name: resolved-654adh
    spec:
      class: olm.resolveset
      refs:
      - file://content
      volumeMounts:
      - mountPath: /content
        configMap:
          name: resolved-654adh-content
          namespace: olm

## ProvisionerClass API

ProvisionerClass defines a configuration for a provisioner. Provisioners are `controllers` that understand `Instance`, `Bundle`, and `ProvisionerClass` APIs and take action.

Each `provisioner` has a unique id. For example, the provisioner that ships with OLM and understand operator bundles has the id: `operators.coreos.com/operators`.

A `ProvisionerClass` specifies a specific configuration of provisioning (i.e. settings to interpret `Instance` and `Bundle` apis). A `provisioner` will only operate on `Instance` and `Bundle` objects that reference a `ProvisionerClass` that contain the `provisioner`'s unique id.

If this seems familiar, it is the same pattern that is used by `StorageClass` / `PeristentVolume` / `PersistentVolumeClaim` in kubernetes.

Example `ProvisionerClass`:

kind: ProvisionerClass
apiVersion: rukpak.io/v1
metadata:
  name: <good name>
provisioner: <unique id>
# parameters has no schema, is provisioner-specific
parameters: {}

### ProvisionerClass Permission

`ProvisionerClass` comes with a `use` verb.

Only users and serviceaccounts with `use` on a `ProvisionerClass` can create an object that references that `ProvisionerClass`.

A webhook rejects requests on `Instance`, and `Bundle` that reference `ProvisionerClass`es that the requesting user/serviceaccount does not have the `use` verb on.

## Approval

Approval is an implementation detail of provisioners, which may choose to block rollout until certain conditions are met.

Instances have an `Approved` condition:

kind: Instance
metadata:
  name: resolved-654adh
spec:
  # omitted
status: 
  conditions:
  - lastUpdateTime: "2020-02-08T11:37:35Z"
    lastTransitionTime: "2020-02-08T11:37:35Z"
    message: Approved by my custom approver controller
    reason: ApprovedByMyPolicy
    status: True
    type: Approved

A webhook ships with RukPak that prevents any update to the `Approved` condition by any user or service account that lacks the `approve` verb on the `Instance` API.

The provisioner for the specified provisioner class is responsible for determining if approval is required or not. The provisioner may choose to auto-approve by writing the `Approved` condition into the status, or it may choose to leave the `Instance` unapproved. A user, or a separate approval plugin, may take over the approval from there.

`kubectl instance approve <my-instance>` will approve an instance that is awaitng approval, as long as the user has sufficient permission.

---
---
---

# Deppy

Deppy runs on- or off-cluster to provide dependency resolution for catalogs of RukPak bundles.

Deppy provides an API for resolving constraints over catalog content (see: [declarative index format](https://github.com/operator-framework/enhancements/blob/master/enhancements/declarative-index-config.md)).

The on-cluster installation of Deppy also includes:

- Continual (optional) resolution against catalog content, to keep installed packages up-to-date.
- A RukPak `provisioner` and default `ProvisionerClass` for `deppy.resolveset` bundles.
- An API for defining constraints at runtime (`Input`)
- A RukPak Approval plugin that provides Android-style update policy

## ResolveSet Bundles

This is an example of a `Bundle` for a local `deppy.resolveset` bundle. It declares the the contents specified by `refs` are needed locally by other consumers in the cluster.

The controller watching this `Bundle` decides how to unpack and is configured via `spec.class`. In this case, the provisioner knows that the `resolveset` lists out other `Bundles` that should be created on the cluster, and begins to create them.

kind: Bundle
metadata: 
  name: etcd-operator.v0.9.3
spec:
  class: deppy.resolveset
  refs:
  - file://content
  volumeMounts:
  - mountPath: /content
    configMap:
      name: resolved-654adh-content
      namespace: olm
status:
  unpacked: NotStarted | InProgress | Done
  objects:
  - /objects/resolved.bundles.json
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: resolved-654adh-content
  namespace: default
data:
  resolved.bundles.json: |-
   {
    "schema": "bundle.v1",
    "packageName": "quay.io/operatorhubio/etcd",
    "path": "quay.io/operatorhubio/etcd:v0.6.1",
    "version": "0.6.1",
    "properties": [
        {
            "name": "pivotFrom",
            "value": "etcd-v0.6.0"
        },
        {
            "name": "olm.gvk",
            "value": {
              "group": "etcd.database.coreos.com", 
              "version": "v1beta2",
              "kind": "EtcdCluster"
            }
        },
    ],
    "channels": [
        "alpha"
    ],
   },
   {
      "schema": "bundle.v1",
      "packageName": "quay.io/operatorhubio/prometheus",
      "version": "1.0.0",
      "properties": [
        {
          "name": "olm.label",
          "value": "LTS",
        }
      ]
   }

## ResolveSet Provisioner Configuration

Deppy ships with one provisioner, `deppy.io/resolver`, that understands `resolveset` bundles and knows how to unpack, apply, and pivot them.

The default `ProvisionerClass`es configure one of these provisioners to support common `Instance` and `Bundle` workflows.

kind: ProvisionerClass
apiVersion: rukpak.io/v1
metadata:
  name: olm.resolveset
provisioner: operators.coreos.com/olm
parameters:
  approval: AllowAll | DenyAll | Android
  # for a given bundle entry, determine what ProvisionerClass to use when stamping out a Bundle
  matchBundle:
  - selector: ".schema == olm.bundle"
    class: olm.bundle
  - selector: ".schema == helm.chart"
    class: helm.bundle
  # equivalent to above when picking ProvisionerClass based on schema
  matchSchema:
  - schema: olm.bundle
    class: olm.bundle

### RukPak Approval Plugin

The resolveset provisioner understands the folliowing parameter in `ProvisionerClass`

kind: ProvisionerClass
parameters:
  approval: AllowAll | DenyAll | Android

- `AllowAll`: Any Instance for a Bundle using a ProvisionerClass with this setting will be created with the `Approved` condition.
- `DenyAll`: Any Instance for a Bundle using a ProvisionerClass with this parameter will be created without the `Approved` condition. Another actor (human, automation) can add `Approved` after the fact.
- `Android`: Any Instance for a Bundle using a ProvisionerClass with this parameter will be created with the `Approved` condition if it can be determined that no bundle exceeds the permission of the previously installed bundle.

The value of `approval` for the default `deppy.resolveset` `ProvisionerClass` is `Android`.

## Input API

The `Input` APIs are used to create inputs for the resolver. An instance of `Input` is more or less an `Installable` (in internal resolver parlance).

apiVersion: deppy.io/v1
kind: Input
metadata:
  name: plumbus
spec:
  inputClass: subscription
  constraints:
  # most common - equivalent to a Subscription
  - type: olm.catalog
    value: 
      name: community
      namespace: my-ns
      package: plumbus
      channel: stable

  # other examples of constraints - less common, but useful

  # lock to a specific version by package/version
  - type: olm.packageVersion
    value:
      package: "plumbus"
      version: v2.0.0

  # lock to a specific version by name
  - type: olm.name
    value: "plumbus.v2.0.0"

  # restrict to a range of versions
  - type: olm.packageVersion
    value:
      package: "plumbus"
      version: ">2.0.0 <3.0.0"

  # arbitrary rego queries
  - type: olm.rego
    value: "semver.Compare(minOCPVersion, 4.8.0) == 1"

status: 
  # instances with properties that match the constraints
  instances:
  - name: plumbus.v2.0.0-alpha
    # other instances satisfied by this instance
    satisfies: 
    - kind: Instance
      name: other-operator
      apiVersion: rukpak.io/v1
      meets:
      - {"olm.constraint": {"olm.name": "etcd-operator.v0.9.3"}}
    # dependencies introduced by this instance
    dependencies:
    - kind: Instance
      name: prometheus-operator-abc
      apiVersion: rukpak.io/v1
      meets:
      - {"olm.constraint": {"olm.gvk": {"group": "manufacturing.how.theydoit.com", "version": "v1alpha1", "kind": "Grumbo"}}
      - {"olm.constraint": {"olm.label": "LTS"}}
    # isntance-specific conditions
    conditions:
    - type: DependenciesMissing
      status: True
      reason: Only 2/3 dependency constraints are met.
      message: "etcd-operator has the following constraints: X,Y,Z. X and Z are satisfied by prometheus-operator-abc, but no instance satisfies Y"
      lastTransitionTime: "2019-09-16T22:26:29Z"
  
  # conditions about the input itself
  conditions:
  - type: ResolutionFailed
    message: "unable to find a solution that matched constraints: X requires Y, but Z requires !Y, X and Z are mandatory"
    status: True

## InputClass API

InputClass defines a configuration for an `Input`.

End-users may select, but will likely never write, an `InputClass`.

### Default InputClasses

Deppy ships with a set of default InputClasses

#### Simulate `Subscription` behavior

kind: InputClass
apiVersion: deppy.io/v1
metadata:
  name: subscription
parameters:
  # if force = true, all matching entries will be installed 
  # directly, and the resolver will not be engaged
  force: false
  # if true, new `Inputs` will be generated for any Instances
  # that have been installed as a dependency
  generateInputsForDependencies: true
  # if true, does not output an `Instance`
  dryRun: false
  # if true, the resolveset bundle includes a lot of data about the resolution process
  trace: false
  # any constraints defined here are included automatically in the `Input` constraints that reference this class
  constraints: 
  - type: minKubeVersion 
    value: >= 1.20
  - type: minOCPVersion
    value: >= 4.7

#### Force install operators

kind: InputClass
apiVersion: deppy.io/v1
metadata:
  name: force
parameters:
  force: true
  generateInputsForDependencies: false
  dryRun: false
  trace: false
  constraints: 
  - type: minKubeVersion 
    value: >= 1.20
  - type: minOCPVersion
    value: >= 4.7

#### Install dependencies, but don't keep them up-to-date independently (i.e. minimal version selection)

kind: InputClass
apiVersion: deppy.io/v1
metadata:
  name: minimal-version
parameters:
  force: true
  generateInputsForDependencies: false
  dryRun: false
  trace: false
  constraints: 
  - type: minKubeVersion 
    value: >= 1.20
  - type: minOCPVersion
    value: >= 4.7

#### Resolve but don't apply

kind: InputClass
apiVersion: deppy.io/v1
metadata:
  name: dry-run
parameters:
  force: false
  generateInputsForDependencies: false
  dryRun: true
  trace: false
  constraints: 
  - type: minKubeVersion 
    value: >= 1.20
  - type: minOCPVersion
    value: >= 4.7

kind: InputClass
apiVersion: deppy.io/v1
metadata:
  name: dry-run-next
parameters:
  force: false
  generateInputsForDependencies: false
  dryRun: true
  trace: false
  # set to the next minor release of kube/ocp 
  constraints: 
  - type: minKubeVersion 
    value: >= 1.21
  - type: minOCPVersion
    value: >= 4.8

## CatalogSource API

Just the normal CatalogSource API. Cluster scoped?

## Resolver API

Note: See Future Work notes at the end of the doc for caveats / considerations.

The resolver has an interactive (**non-kube**) API that supports:

### Queries

Input:
- a constraint
- (or) a rego query

Output:
- a list of matching bundles from the catalog cache

### Interactive Resolution

Input:
- a list of lists of constraints (i.e. a list of Inputs)

Output:
- resolved set, or error

---
---
---

# Operator Provisioner

The Operator provisioner understands `registry+v1` manifest bundles and instances, providing support for `olm.bundle` entries in catalogs. It also comes with:

- Default ProvisionerClasses for operator bundles
- An approval plugin for operator bundles
- A set of default `InputClass`es for use when installing operator bundles
- A porcelein `Operator` API that can be used for installing operator bundles in lieu of writing `Input` directly.

### Operator Bundle Example

This is an example of a `Bundle` for a remote `registry+v1` bundle image. It declares the the contents specified by `refs` are needed locally by other consumers in the cluster.

The controller watching this `Bundle` decides how to unpack and is configured via `spec.class`. In this case, the provisioner knows that this is `registry+v1` bundle, and unpacks the `manifests` held within.

The `status` is controlled by the `provisioner` configured on the `class`, but it lists out locations in the cluster that unpacked content can be found.

kind: Bundle
metadata: 
  name: etcd-operator.v0.9.3
spec:
  class: olm.bundle
  refs: 
  - docker://quay.io/etcd/bundle@sha256
status:
  unpacked: NotStarted | InProgress | Done
  objects:
  - /objects/csv
  - /objects/role

### Operator Bundle Instance Example

This is an example of an `Instance` that declares that `registry+v1` bundle image should be pulled down and installed on the cluster.

The `selector` may match other (older) bundles. The controller watching this instance decides how to pivot and is configured via `spec.bundle.spec.class`, or on `spec.class` of the `Bundle` object that `Instance` points to.

kind: Instance
spec: 
  # selector is used to find previous/current Bundles
  selector: 
    matchLabels:
      subscription: etcd-operator
  # a reference to the Bundle object that should be considered "Active".
  bundle: 
     name: etcd-operator.v0.9.3
     # optional, for direct application of a bundle definition
     spec:
       class: olm.bundle
       refs: 
       - docker://quay.io/etcd/bundle@sha256
       # or: volume-backed
status:
  conditions:
  - type: Approved
    status: True
    reason: ApprovedByMyPolicy
    message: Approved by my custom approver controller
    lastTransitionTime: "2020-02-08T11:37:35Z"

### Default ProvisionerClasses

The default `ProvisionerClass`es configure one of these provisioners to support common `Instance` and `Bundle` workflows.

kind: ProvisionerClass
apiVersion: rukpak.io/v1
metadata:
  name: olm.bundle
provisioner: operators.coreos.com/operators
parameters:
  approval: AllowAll | DenyAll | UserPermission
  # options for unpacking bundles
  maxObjectSize: 10mb

#### Approval for the operator provisioner

The operator provisioner understands the folliowing parameter in `ProvisionerClass`

kind: ProvisionerClass
parameters:
  approval: AllowAll | DenyAll | UserPermission

- `AllowAll`: Any Instance for a Bundle using a ProvisionerClass with this setting will be created with the `Approved` condition.
- `DenyAll`: Any Instance for a Bundle using a ProvisionerClass with this parameter will be created without the `Approved` condition. Another actor (human, automation) can add `Approved` after the fact.
- `UserPermission`: Any Instance for a Bundle using a ProvisionerClass with this parameter will be created with the `Approved` condition if it can be determined via `SubjectAccessReview` checks that the user that created it has sufficient permission to create all of the contents of the Bundle.

The value of `approval` for the default `olm.bundle` `ProvisionerClass` is `UserPermission`.

### Operator API

The simplest way to install operators and subscribe to updates is via the `Operator` API. The current status and install progress, as well as references to all components of an operator, can be found on the `status` block of the `Operator` object.

Example:

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: subscription
  
  # optional
  # this field is translated directly to an `Input` constraint
  # that looks like:
  # 
  # type: olm.catalog
  # value: 
  #   name: community
  #   namespace: my-ns
  #   package: plumbus
  #   channel: stable
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable
    
  # optional, lock to a specific version
  # this field is translated directly to an `Input` constraint
  # that looks like:
  # 
  # type: olm.packageVersion
  # value: 
  #   package: plumbus
  #   version: v2.0.0
  # 
  version: "2.0.0"
  
  # version may also be a semver range, like ">2.0.0 <3.0.0" 
  version: ">2.0.0 <3.0.0"
  
  # optional
  # additional arbitrary constraints can be added 
  constraints:
  - type: olm.rego
    value: "semver.Compare(minOCPVersion, 4.8.0) == 1"

status:
  updates:
  - options:
    - plumbus.v2.0.8
    - plumbus.v2.0.7
    - plumbus.v2.0.6
    catalogSourceRef:
      name: community
      namespace: my-ns
    package: plumbus
    channel: beta

  conditions:
  - kind: UpdateAvailable
    status: True
    reason: CrossChannelUpdateFound
    message: updates have been found that require a change in the operator spec to apply
    lastTransitionTime: "2019-09-16T22:26:29Z"

  components:
   matchLabels:
      operators.coreos.com/plumbus: ""
   refs:
    - kind: Input
      name: plumbus
      apiVersion: rukpak.io/v1
    - kind: Instance
      name: plumbus.v2.0.0-alpha
      apiVersion: rukpak.io/v1
      conditions:
      - type: Installing
        status: true
    - kind: Bundle
      name: plumbus.v2.0.0-alpha
      apiVersion: rukpak.io/v1
      conditions:
      - type: Unpacked
        status: true  
    # older bundles visible during pivot
    - kind: Bundle
      name: plumbus.1.9.0
      apiVersion: rukpak.io/v1
      conditions:
      - type: Unpacked
        status: true
      - type: Replaced
        status: true
    - kind: ClusterServiceVersion
      namespace: operators
      name: plumbus.v2.0.0-alpha
      apiVersion: rukpak.io/v1alpha1
      conditions:
      - type: Installing
        status: True
        reason: AllPreconditionsMet
        message: deployment rolling out
        lastTransitionTime: "2019-09-16T22:26:29Z"
    - kind: CustomResourceDefinition
      name: plumbai.how.dotheydoit.com
      apiVersion: apiextensions.k8s.io/v1beta1
    - kind: ClusterRoleBinding
      namespace: operators
      name: rb-9oacj
      apiVersion: rbac.authorization.k8s.io/v1

When an `Operator` is created, an appropriate `Input` will be created. The `Operator` controller makes use of the Resolver API to determine a set of potential updates to highlight in the status (i.e. by querying in other channels / relaxing constraints).

Most users will not need to look at other APIs.

# Future Work: Constraints, MetaConstraints, and Preference

This topic has not been nailed down to a degree that a strawman made sense. Instead, the scope of the problem is outlined below.

There is a big limitation in the current model (in `deppy`, specifically), which limits the constraints and meta-constraints that can be specified as input.

For example, this constraint:

type: olm.catalog
value: 
  name: community  
  namespace: my-ns
  package: plumbus
  channel: stable

Gets translated to a boolean formula for the sat-solver that looks like:

[A or B or C or D]

Where `A`, `B`, `C`, and `D` are all of the operators in the `stable` channel for the `plumbus` package. This constrains all solutions to only those that can be found in the specific package/channel, encoding it in a way that the solver understands.

Today OLM does the translation of constraints to these boolean formulas (we call a single formula an "installable" in the OLM codebase).

The resolver also has a set of constraints that enforce properties and invariants about the solution. Today these are things like "only one subscription to a package can exist in a channel" or "only one operator can provide a gvk in a namespace".

These "meta-constraints" are also translated into boolean formulas and fed to the solver.

In the model above, each new constraint type or constraint needs to be encoded in `deppy`, because `deppy` does the translation of constraints to solver inputs (boolean formulas).

We know that, when building a package ecosystem, it is useful to be able to define this type of constraint. We would like to be resilient to future extension needs (i.e. it should be straightforward to add helm-specific constraint types, as the need arises).

The `rego` constraint type gives us some flexiblity, but limits us to filtering that can be expressed as a rego query.

Some options we have discussed:
- Constraints should all be specified as rego queries
- Define plugin system that can be called directly from deppy (wasm?) for filtering
- Define an InputClass "provisioner" that provides an API that converts from a constraint to a boolean formula
- A `Solve` API that takes as input boolean constraints. Plugins translate constraints on `Input` into boolean constraints on `Solve`, the resolver outputs its results into the status of `Solve.`

Preference is also not currently pluggable.

Preference is the property in the resolver that tests solution in a particular order - it's the reason that an operator from the head of a channel is picked instead of an older release, if it can be. It's the reason operators from the redhat-catalog are preferred to operators in the community-catalog, and why dependencies are preferred from the same catalog from which they came.

As we expand to other artifact types, it's clear that we would like to be able to define preference order for other constraints.

# Changelog
- 0.6.2
- migrated doc ownership

- 0.6.1
- move satisfies/dependency info to Inputs
- add back in user stories
- renamed depster -> deppy
- 0.6
- cleanup
- clarify pluggability questions / future work
- 0.5
- One project for instance / bundle (rukpak)
- One project for resolver + resolver apis (depster)
- Plugins for those
- 0.4
- Separate operator-specific things from resolver+bundle things
- 0.3
- Added an arch diagram
- Named resolver+bundle subproject `RukPak` with new group `rukpak.io`
- `Operator` API puppets `Input` API for now
- Error messages on `Input` API
- Adds option for tracing resolver output, added to resolveset
- Add sketch of Resolver API to answer questions about available updates / dry-run
- Added a note about future extension via `InputMetaClass`
- 0.2
- add an FAQ
- add a new API (`Input`) for constraints, separate from Operator
- re-usable across other artifact types
- factor out the top-level fields in Operator
- entrypoint when installing somethign that is not an Operator
- make the `Approved` condition match other condition apis
- 0.1
- initial e2e: do we get everything we need with just Operator + Bundle + Instance?

---
---
---

# User Stories

Evaluate the design against target user stories.

### Force install a single operator from a catalog: don't subscribe it to updates, don't resolve dependencies

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: force
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable
    
  version: "2.0.0"

- Creates an `Input` with matching constraints
- Creates an `olm.resolveset` Instance that contains every bundle that matches the constraints
- This may pick multiple bundles
- This does not fulfil dependencies - the resolveset contains only the bundles that match the query

### Install an operator and its dependencies from a catalog but don't subscribe it to updates.

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: minimal-version
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable
  version: "2.0.0"  

- Creates an `Input` with matching constraints
- Creates a `resolveset` Instance
- Creates a set of Instances for all bundles
- Constraints lock this to a particular version, no updates occur

### Install a single operator from a catalog and subscribe it to updates, but don't subscribe its dependencies to updates.

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: minimal-version
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable

- Creates an `Input` with matching constraints
- Creates a `resolveset` Instance
- creates a set of Instances for all bundles
- Constraints allow for updates

### Install an operator and its dependencies and subscribe all of the operator and all of its dependencies to updates.

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: subscription
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable

- Creates an `Input` with matching constraints
- Creates a `resolveset` Instance
- creates a set of Instances for all bundles
- For each Instance in the `resolveset`, creates an `Input` with appropriate constraints

### Install a single operator bundle directly (no catalog)

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:  
  bundle:
    name: etcd-operator.v0.9.3
    spec:
      class: olm.bundle
      refs: 
      - docker://quay.io/etcd/bundle@sha256

which results in:

kind: Instance
spec: 
  # selector is used to find previous/current Bundles
  selector: 
    matchLabels:
      subscription: etcd-operator
  # a reference to the Bundle object that should be considered "Active".
  bundle: 
     name: etcd-operator.v0.9.3
     # optional, for direct application of a bundle definition
     spec:
       class: olm.bundle
       refs: 
       - docker://quay.io/etcd/bundle@sha256
       # or: volume-backed
status:
    # progress, failure, etc
    # 

### Update the dependencies of an operator even if there is no "subscription" for the dependencies.

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: minimal-version
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable

- Creates an `Input` with matching constraints
- `minimal-version` means that the dependencies will only update when this one has requirements on newer versions.

### Force update a single operator directly to a specific version, with no catalog available in the cluster.

Assuming there is an existing `Instance` on the cluster for `etcd-operator`:

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:  
  bundle:
    name: etcd-operator
    spec:
      class: olm.bundle
      refs: 
      - docker://quay.io/etcd/bundle@sha256

will result in:

kind: Instance
spec: 
  # selector that matches the existing Bundle we're updating
  selector: 
    matchLabels:
      subscription: etcd-operator
  # a reference to the Bundle object that should now be considered "Active".
  bundle: 
     name: etcd-operator
     spec:
       class: olm.bundle
       # ref to the bundle we want to update to
       refs: 
       - docker://quay.io/etcd/bundle@sha256
status:
    # progress, failure, etc
    # 

- This attempts to directly update an installed bundle to another bundle, pivoting ownerrefs, etc

### Force update a single operator directly to a specific version from a catalog

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: force
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable
    
  version: "2.0.0"

- Creates an `Input` with matching constraints
- Creates an `olm.resolveset` Instance for every bundle that matches the constraints, with appropriate "pivot" data added.

### Able to change the properties or constraints of any installed bundle.

Properties and constraints, as determined by the resolver, are persisted as annotations on the Bundle. They can be edited by a user to change the way resolver sees them (this is generally inadvisable and should only be done when absolutely necessary).

kind: Bundle
apiVersion: rukpak.io/v1
metadata:
  name: plumbus-kjf122
  annotations:
    olm.properties: -|
     {"olm.gvk": {"group": "how.theydoit.com", "version": "v2alpha1", "kind": "Plumbus"}}
     {"olm.constraint": {"olm.gvk": {"group": "manufacturing.how.theydoit.com", "version": "v1alpha1", "kind": "Grumbo"}}
spec:
 class: registry.v1
 refs: 
 - docker://quay.io/plumbus/bundle@sha256:abcdef12345
status:
    unpacked: Done
    objects:
    - /objects/csv
    - /objects/role

### Communicate to the user when the things that are already installed have unsatisfied dependencies and communicate to a user that dependencies are satisfied by specific other bundles.

Status is provided on Input. This is aggregated to the `Operator` status via existing mechanisms.

apiVersion: depster.io/v1
kind: Input
metadata:
  name: plumbus
spec:
  inputClass: subscription
  constraints:
  - type: olm.catalog
    value: 
      name: community
      namespace: my-ns
      package: plumbus
      channel: stable
status: 
  # instances with properties that match the constraints
  instances:
  - name: plumbus.v2.0.0-alpha
    # other instances satisfied by this instance
    satisfies: 
    - kind: Instance
      name: other-operator
      apiVersion: rukpak.io/v1
      meets:
      - {"olm.constraint": {"olm.name": "etcd-operator.v0.9.3"}}
    # dependencies introduced by this instance
    dependencies:
    - kind: Instance
      name: prometheus-operator-abc
      apiVersion: rukpak.io/v1
      meets:
      - {"olm.constraint": {"olm.gvk": {"group": "manufacturing.how.theydoit.com", "version": "v1alpha1", "kind": "Grumbo"}}
      - {"olm.constraint": {"olm.label": "LTS"}}
    # isntance-specific conditions
    conditions:
    - type: DependenciesMissing
      status: True
      reason: Only 2/3 dependency constraints are met.
      message: "etcd-operator has the following constraints: X,Y,Z. X and Z are satisfied by prometheus-operator-abc, but no instance satisfies Y"
      lastTransitionTime: "2019-09-16T22:26:29Z"

### Communicate to the users when they are about to install operators, which dependencies would be resolved

- Installing new content by default will result in an Instance waiting to be approved (due to android permissions)
- User reviews the content of the instance before `kubectl instance approve`
- Users that want more control can use a provisionerClass with `DenyAll` so that every single change requires review.

### Don't block updates to operators when other unrelated operators are blocked.

The resolver will generate disjoint resolveset `Instance`s.

### Understand why an update is blocked

The resolver will emit events in the namespace.

Most other states should be readily visible in current `Input` (and `Operator`) status

### If an operator is not subscribed to updates, be presented with options for update as they become available (OCP-style).

- If using the `Operator` API, the status will include information about potential updates
- The resolver API can be queried with constraints in the general case.

### Operator updates available in other catalogs or channels are surfaced to users.

- If using the `Operator` API, the status will include information about potential updates
- The resolver API can be queried with constraints in the general case.

### Ability to write policy for updates. Andriod-style permission policy comes by default.

Instances may have an `Approved` condition:

kind: Instance
metadata:
  name: resolved-654adh
spec:
  selector: 
    matchLabels:
      subscription: etcd-operator
  bundle:
    name: resolved-654adh
    spec:
      class: olm.resolveset
      refs:
      - file://content
      volumeMounts:
      - mountPath: /content
        configMap:
          name: resolved-654adh-content
          namespace: olm
status: 
  conditions:
  - lastUpdateTime: "2020-02-08T11:37:35Z"
    lastTransitionTime: "2020-02-08T11:37:35Z"
    message: Approved by my custom approver controller
    reason: ApprovedByMyPolicy
    status: True
    type: Approved

Depster ships with approval plugins as part of the `olm.resolveset` provisioner, the operator provisioner ships with approval plugins for `olm.bundle`. Additional plugins can be written, configured, and installed as needed.

### Non-administrators should be able to install non-operator content, as long as they have sufficient permission for the install.
Strawman: As a non-administrator user, I would like to trigger a namespace-local installation of a helm chart via a bundle.

A helm provisioner is installed that watches `Instance`, and `Bundle`. It may also choose to include porcelein APIs that translate to `Inputs`

When I create an `Instance` that contains references a provisionerclass that points to the helm provisioner, the helm provisioner is used to install.

The helm provision may choose to provide additional apis for visibility or control (just as olm provides the `Operator` API).

The helm provisioner may provide a `HelmChart` API that is namespaced-scoped and performs access checks for a user before creating an `Input` on a user's behalf.

Strawman 2: An additional namespace-scoped API can be added, `OperatorRequest`, that either results in the creation of an unapproved `Instance`

### As a non-administrator user, I would like to request an Operator install in the style of Kubernetes certificate signing requests getting approved by admins

This should more or less be covered by the approval workflow + a way to request installation with namespace scoped APIs.

### Determine what would be installed under specific conditions - i.e. if the cluster were updated to a new version, if a specific operator is selected

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: dry-run
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable
    
  version: "2.0.0"

results in

apiVersion: rukpak.io/v1
kind: Input
metadata:
  name: plumbus
spec:
  inputClass: dry-run
  constraints:
  - type: olm.catalog
    value: 
      name: community
      namespace: my-ns
      package: plumbus
      channel: stable
  - type: olm.packageVersion
    value:
      packageName: plumbus
      version: "2.0.0"

The `dry-run` inputClass sets `dryRun: true`. When dry-run is true, the resolver will output a resolveset `Bundle` but not a resolveset `Instance`, so no changes occur to running bundles.

Alternatively, the resolver API can be queried interactively for this information.

### If a dependency is in a degraded state, I can tell this from the top-level operator.

The `Operator` aggregates status of its components, which includes `Input` objects, which report dependency (satisfaction) problems.

Actual dependency health: TODO

### Install at a particular version and then enable updates.

1. Create an `Operator` to install at a specific version

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: minimal-version 
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable
    
  version: "2.0.0"

2. Edit the `Operator` to enable updates

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: minimal-version 
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable

### Install at a particular version, and walk through a set of updates, stopping before hitting latest.

1. Create an `Input` to install at a specific version

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: minimal-version 
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable
    
  version: "2.0.0"

2. Edit the `Input` to enable updates up to a specific release

apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  name: plumbus
spec:
  inputClass: minimal-version 
  catalog:
    name: community
    namespace: my-ns
    package: plumbus
    channel: stable
    
  version: "<= 2.4.5"

## Install two operators that own the same APIs

## Adjacent stories

Stories that are best addressed with features outside the scope of this enhancement.

### View the update graph for a particular operator in the console.

The package server can be queried for this data, the console can display it.

### Present a user with release notes for available updates, and with release notes for currently installed operators.

Given any `bundle`, the package-server can be queried for release notes.

Detailed release notes or docs may be part of an operator's bundle, visible after install.

### Non-administrators can get info about operator-provided APIs based on their access to them.

Options:

- Hack into discovery / openapi
- Have a cluster-scoped api just for this info
- Add extended property support to kube
- "man pages" for kube