[SRVKP-2389] — Simplify security configurations for image builds

# [SRVKP-2389] — Simplify security configurations for image builds ###### tags: `tektoncd` `openshift-pipelines` `pipeline` `tep` # Summary > As an admin, I want a simple way to configure the security level with regards to building images in OpenShift Pipelines, so that I can configure the platform in accordance to the security requirements of our organisation. > > Buildah requires special permission for building images on OpenShift and as a result, the OpenShift Pipelines performs a series of configurations (pipeline service account, pipeline-scc, etc) in order to enable image builds to work out of the box. These configurations are not desirable for all customers and some would prefer to disable some or all of these configurations that are made for buildah. Nevertheless, discovering and applying these configurations is a daunting task and requires changes in many places including the Task specs and SCC. > > The currently supported security levels for building images are: > * buildah runs as root in a privileged pod: +all+ image builds work > * buildah runs as root, unprivileged with pipelines-scc (default): +majority+ of image builds work > * buildah runs as root in the user namespaces with pipelines-scc: +most+ image builds work > * buildah runs as user build with custom scc run as build: +most+ image builds work > * No special treatments, sa, or scc: +none+ of image builds work # Design ## General Ideal (ideas, not real yet) - OpenShift Pipelines ships with only `default` SA (not creating any), restricted SCC… - Same as upstream - Secure by default (installing OSP doesn't add new privileges to any users) - A ClusterAdmin can set the higher level one can set through namespace and a default level to be applied - This allow cluster admins to control "exposition" without having to manage all namespace themselves - A namespace requesting a level higher than the allowed, generates errors (and namespace events to help diagnose what is happening) - We would start the default level to be similar level to what we ship today (1.8), and we could decide to lower this later on - A user can annotate a namespace to "grant" that namespace with an SA that has additionnal *privileges* to build images - TODO: define the list of levels (probably from above more or less) - Scoped to namespace - Only namespace admin can configure this And of course **heavily document this** :angel:. ## Levels > - pipeline sa exists. buildah runs as root in a privileged pod > - pipeline sa exists, buildah runs as root, unprivileged with pipelines-scc (default) > - pipeline sa exists, buildah runs as root in the user namespaces with pipelines-scc > - pipeline sa exists, buildah runs as user build with custom scc > - pipeline sa exists, runs with restricted scc > - pipeline sa and custom scc DON'T exist Here the proposed list of levels we would want to go for, from the highest (more privileges to the lowest (less privileges). 1. Level 4: `pipeline` SA exists with `privileged` SCC : can run pods (buildah) as `root` and as `privileged` 2. Level 3: `pipeline` SA exists with `anyuid`-like SCC: can run pods (buildah) as `root` but not `privileged`. - can run as `root` on the container and host - can run as `root` on the containerd and user on the host 3. Level 2: `pipeline` SA exists with `nonroot`-like SCC: can run pods (buildah) as a specific user id (`1000` for buildah image) but not as `root`. - cannot run as `nonroot` - must run as `uid=1000` 4. Level 1: `pipeline` SA exists with `restricted`: cannot run buildah, similar to `default` SA 5. Level 0: `pipeline` SA doesn't exist For each of those levels, we would need documentation on how to use and run builds, especially building images. Also the documentation should help the end-user, namespace admins and cluster admins to decide which levels fits the most for their needs. ## Configuration ### Cluster Admin ```yaml # […] kind: TektonConfig # […] spec: # […] security: defaultLevel: 2 # nonroot maxLevel: 3 # anyuid ``` Default level is `nonroot` (a `pipeline` SA with a `nonroot`-like scc but that runs as `uid=1000`). And one namespace cannot request more than level3 (aka can request for `anyuid`-like but not level4, aka `privileged`). ### Namespace ```yaml # […] kind: Namespace metadata: name: foo annotations: operator.tekton.dev/security-level: 1 # restricted ``` ```yaml # […] kind: Namespace metadata: name: bar annotations: operator.tekton.dev/security-level: 4 # privileged ``` This would fail if the `maxLevel` is lower. The way to see the failure should be on the Namespace Events. ## Levels "configuration" In all cases, we ask the user to go through our resolver (`cluster`, `tomato` or `osp`, *name to find*) to get the `buildah` spec. This allow us to return a different version of it depending on the security level configuration. ### Level 0 : no pipeline SA - No `pipeline-scc` is created - No `pipeline` SA is created This is equivalent to `createRBACResources: false` today. ### Level 1 : restricted - No `pipeline-scc` is created - `pipeline` SA is created with `restricted` ### Level 2 : nonroot - `pipeline-scc` is created with a `nonroot`-like content (force uid to 1001) - `pipeline` SA is created with `pipeline-scc` - `buildah-nonroot` to be create/used (run as user `build`, chroot, vfs) ### Level 3 : anyuid - `pipeline-scc` is created with a `anyuid`-like content (force uid to 1001) - `pipeline` SA is created with `pipeline-scc` - `buildah-root` to be create/used (run as `root`, chroot, vfs) This is equivalent to today's approach when `createRBACResources: true`. ### Level 4 : privileged - no `pipeline-scc` is created - `pipeline` SA is created with `privileged` - `buildah-privileged` to be create/used (run as `root`, chroot or containers, overlay) # Alternatives & explorations ## Tailored SCC per "image" or "group of task" In a gist, OpenShift Pipelines could ship with a set of pipeline specific SCC (like "run as uid 1000 for buildah", "run as uid 65532 for git-clone & co", "run as non root", …) and we would attach the SA+SCC on the TaskRun/Pod that would belong to that particular group *or* using that particular image. Pipeline-as-code helps us here (as well as resolvers) as they it allows us to mutate the definition (embedded) live to add a specific SA or specific SCC to a given PipelineTask (and thus TaskRun/Pod). ## Provide # Requirements ## Per-namespace configuration In order for this to be *better* supported, we need [TEP-0085: Per-Namespace controller configuration](https://github.com/tektoncd/community/blob/main/teps/0085-per-namespace-controller-configuration.md) implemented. This would be required for tektoncd/pipeline as well as tektoncd/operator. Note that `tektoncd/operator` can already adopt this "namespace" thingy without the need for tektoncd/pipeline to support it. ## At installation configuration (optional) As of today, we create an instance of OSP at operator installation time. This means any user that has needs that differs from the defaults, have to modify the configuration afterwards, which can, in terms of security, leave them open for a time being. *Note: this is not a problem if the default is `restricted`* An epic tracks this : https://issues.redhat.com/browse/SRVKP-2407 [SRVKP-2389]: https://issues.redhat.com/browse/SRVKP-2389 # References - https://blog.chmouel.com/2022/01/25/user-namespaces-with-buildah-and-openshift-pipelines/ - https://blog.chmouel.com/2022/03/07/running-tasks-as-non-root-on-openshift-pipelines/ - https://frasertweedale.github.io/blog-redhat/posts/2022-02-02-openshift-user-ns-without-anyuid.html - https://docs.google.com/document/d/1tx-y1s2rZ_sOb2Aidy8bF-hBBhhrb-JuW9UQT6Ndkw0/edit