--- title: Oria Operator authors: - "@awgreene" reviewers: - TBD approvers: - TBD creation-date: 2022-07-28 last-updated: 2022-07-28 status: provisional tags: Descoped Operators see-also: - "https://issues.redhat.com/browse/OLM-2158" - "https://docs.google.com/document/d/1jYJ7I5jABEXyd3viCVGeCbqEQB86O0MhwNZGOmUx51M/edit#heading=h.hjsudcynflmv" - "https://hackmd.io/wVfLKpxtSN-P0n07Kx4J8Q?view" - "https://hackmd.io/mnbOutwJRSW3oty-j5la2w" - "https://github.com/kubernetes-sigs/controller-runtime/issues/1652" --- # Oria Operator ## Summary As part of the [Operator Framework](github.com/operator-framework)'s effort to move towards Descoped Operators we must create an on cluster component responsible for generating an operator's requested RBAC in a set of namespaces supplied by a Cluster Administrator. This document will focus on the controller responsible for generating an operator's requested RBAC. ## Motivation To learn more about the motiviation behind Descoping Operators, please review: - OLM's [Descoping Plan](https://hackmd.io/DTVukSyGSkSLwD9WgZsdbw). - The [Scoped Operators Landing Page](https://hackmd.io/luRu5dE2R924qojSeOaugA). In an effort to support the `descoped operator` model, the operator framework will need to provide a controller that facilitates: - The generation of RBAC requested by the operator in the set of namespaces specified by the Cluster Administrator. - The generation of RBAC that allows specific users, groups, and serviceAccounts to interact with an API introduced by an operator. The [Oria Operator Github Repository](https://github.com/awgreene/oria-operator) acts as a Proof of Concept for the operator described in this document. ### Goals **Provide Operator Authors with the means to:** - Define RBAC required by an operator. **Provide Cluster Admins with the means to:** - Create namespaced RBAC requested by the operator in a set of namespaces. - Generate RBAC for subjects to interact with a CRD introduced by an operator. ### Non-Goals - Define how an operator's informers will be established based on available RBAC. Please review [the informers based on RBAC documentation](https://hackmd.io/BZlW_8iERlCOxZJN30sVeQ) if you'd like to learn more about this problem space. ## Proposal This enhancement proposes the creation of the `oria operator`. The primary purpose of the `oria operator` will be to: - Allow Operator Authors to specify clusterRoles they require. - Allow Cluster Admins to bind clusterRoles specified by an operator to one or more namespaces or at the cluster level. - Allow Cluster Admins to grant specific users, groups, and serviceAccounts the ability to interact with a CRD introduced by an operator. To facilitate the stories above, the `oria-operator` will introduce the `scopeTemplate` and `scopeInstance` CRDs. ### ScopeTemplate CRD The scopeTemplate CRD is used to define: - A clusterRole. - Subjects that must be "bound" to the clusterRole. An example of a scopeTemplate CR can be seen below: ```yaml apiVersion: operators.coreos.com/v1alpha1 kind: ScopeTemplate metadata: name: foo spec: clusterRoles: # Any number of clusterRoles can be defined - generateName: fooClusterRole- rules: - apiGroups: - "" resources: - pods - services verbs: - get - list - create - update - delete - patch - watch bindingTemplate: subjects: # Supports multiple subjects, could be limited to one. - kind: ServiceAccount # MVP can support users, groups, and serviceAccounts. name: fooServiceAccount namespace: # Can be required for the MVP, but must be updated based on the namespace of the serviceAccount. status: conditions: - See [metav1.conditions](https://github.com/kubernetes/apimachinery/blob/c5be38573c736f9d126b0bd7b64eb2b5fe9b453f/pkg/apis/meta/v1/types.go#L1448-L1508), could be used to communicate whether a CR is valid or not, permission errors, etc... ``` When reconciling this resource, the `oria-operator` will: 1. Check if any scopeInstance CRs reference the name of the scopeTemplate. -- If a scopeInstance references the scopeTemplate, the clusterRole defined in the `scopeTemplate` will be created if it does not exist. The created clusterRole will include an [ownerReference](https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents) to the`scopeTemplate` CR. -- If no scopeInstance references the scopeTemplate, the clusterRole defined in the `scopeTemplate` will be deleted if it exists. 2. Any errors creating/deleting the clusterRole will be captured in the `scopeInstance`'s status. 3. A scopeTemplate will be reconciled any time a scopeInstance is modified. > Open Question: Should we allow clusterScopedResources to be included? If not, we should place the CR in a failed state before step 1. ### ScopeInstance CRD The `scopeInstance` CRD is used to define the set of namespaces that the RBAC in a ScopeTemplate wil lbe created in. A cluster admin will create the `scopeInstance` CR and will specify: - The name of a `scopeTemplate` which defines the RBAC required by the operator - A set of namespaces that the operator should be scoped to. An example of a scopeTemplate CR can be seen below: ```yaml= apiVersion: operators.coreos.com/v1alpha1 kind: ScopeInstance metadata: name: foo spec: scopeTemplateName: foo # Required field. namespaces: # If the namespaces array is empty, a clusterRoleBinding will be cretaed. - foo # A roleBinding should be created in the foo namespace - bar # A roleBinding should be created in the bar namespace namespaceSelector: fooKey: barValue # Support roleBinding generation in namespaces with given label. status: conditions: - [See metav1.conditions](https://github.com/kubernetes/apimachinery/blob/c5be38573c736f9d126b0bd7b64eb2b5fe9b453f/pkg/apis/meta/v1/types.go#L1448-L1508) ``` When reconciling this resource, the `oria-operator` will: 1. Search for the referenced `scopeTemplate`. The status of the `scopeInstance` CR will be updated if the `scopeTemplate` is not found or in the failed state. 2. The controller will check the namespaces field: -- If the `namespaces` array is empty, a single clusterRoleBinding will be created. These resources will include an [ownerReference](https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents) to the`scopeInstance` CR. -- If the `namespaces` array is not empty, a roleBinding will be created in each of the `namespaces`. These resources will include an [ownerReference](https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents) to the`scopeInstance` CR. 3. Any failure to create or delete a resource will be reflected in the `scopeInstance`'s status. 4. Any changes to a scopeTemplate will queue scopeInstances that reference the ScopeTemplate. ### Supported Workflows With these CRDs in mind, let's consider a few workflows that we need to cover. #### Operator that Opts into scoping In this scenario, the operator author wants to support scoping by cluster admins, let's see the required workflow: - The operator's informer and cache is dynamically inferred from existing RBAC. - The bundle created by the operator includes a `scopeTemplate` CR which defines the RBAC required by specific serviceAccounts. - After installing the operator, the cluster admin provides a list of namespaces in the `scopeInstance` CR. - The `oria-operator` will then: -- Create the clusterRole defined in the `scopeTemplate`. -- Create the roleBindings or the clusterRoleBinding required by the operator in the set of namespaces defined in the `scopeInstance` CR. - The operator is effectively scoped to a set of namespaces. #### Operator that does not opt into scoping In this scenario, the operator author does not want to support scoping by cluster admins, let's see the required workflow: - The operator is configured to a [single predefined namespace](https://sdk.operatorframework.io/docs/building-operators/golang/operator-scope/#watching-resources-in-a-single-namespace), a [set of predefined namespaces](https://sdk.operatorframework.io/docs/building-operators/golang/operator-scope/#watching-resources-in-a-set-of-namespaces), or [all namespaces](https://sdk.operatorframework.io/docs/building-operators/golang/operator-scope/#watching-resources-in-all-namespaces-default). - The bundle created by the operator does not include a `scopeTemplate` CR, it instead includes the roles, roleBindings, clusterRoles, and clusterRolesBindings required by the operator. - The operator is installed and works as the Operator Author intended. The `oria-operator` is not involved. - Users of the operator must convince the Operator Author to opt into OLM's descoping model. ### User Stories #### Story 1 As a cluster admin, I would like to chose which users can interact with the CRDs owned by the operator I've installed. #### Story 2 As an operator author, I would like to provide a template that cluster admins can use to generate the RBAC my operator needs to reconcile events in specific namespaces. #### Story 3 As an operator author, I would like to provide the RBAC my operator needs to reconcile events in all or a set of namespaces. ## Backwards Compatability In an effort to be backwards compatible, a controller could be introduced that: - Generate a ScopeTemplate for each CSV in a namespace. The list of clusterRoles and subjects could be inferred from the CSV. - Geneate a scopeInstance whose spec.Namespaces field is inferred from the [OperatorGroups](https://olm.operatorframework.io/docs/advanced-tasks/operator-scoping-with-operatorgroups/)'s `spec.TargetNamespaces'. More details can be found in the OLM's [Historical Descoping Plan Transition Plan Section](https://hackmd.io/DTVukSyGSkSLwD9WgZsdbw#Installation-specific-migrations-steps) document. ## Drawbacks - There is no "one click install". Cluster admins need to set operator scope post install or create the `scopeInstance` prior to installing the bundle. ## Alternatives ### Use the Combo Operator instead of the scoping operator Combo templates, while powerful, might be heavy handed for what we need to do.