CBPP: No root in containers / Container should execute process as non-root user

--- tags: best practice proposal, cbpp --- # CBPP: No root in containers / Container should execute process as non-root user ## DRAFT CBPP  [//]: # (This would be a comment that doesn't appear in the output) --- Notes to build from: [google doc notes](https://docs.google.com/document/d/1FQNdbnp-6hlozD5lJf-8JnVoU12fYHQPiEwKCGy-RCw/edit#) Required by: indirectly operator, directly CNF provider (since it improves CNF security) Provided by: CNF developer Detected: how? Straightforward justification Might want to note any downsides (slower dev since access control needs dealing with during container build process Why: - To avoid compromised apps doing even more damage (3) - To avoid uncompromised apps stomping on **Root in containers** - Why it’s used - Root in a container is not immediately threatening - it’s not the system root and it only offers power within the container - so it’s does not affect platform integrity - By default, the first process starting in a container runs as root - you have to actively take steps to shed the permissions (Dockerfile USER, running unprivileged processes - and managing filesystem permissions along with that) - Laziness with regards to handling limited user privileges in a container (eg. root can modify everything w/in the container; it’s easier to use root initially than figure out access details, and in a working app there is no motivation to add security if it threatens to introduce problems) - Why you shouldn’t use it - Some platforms (OpenShift, e.g., to better support use of SELinux) don’t let you have root - Root can change all files in the container image - this can enable compromises because a compromised process has more power than it would otherwise - Since root is not ‘system root’ it doesn’t have significant rights to do things that other UIDs in the container don’t have beyond the above - makes more sense not to permit it - What you can do instead - TBD - (use of low ports) --- - [Release Signoff Checklist](#Release-Signoff-Checklist) - [Summary](#Summary) - [Motivation](#Motivation) - [Goals](#Goals) - [Non-Goals](#non-goals) - [Proposal](#proposal) - [User Stories](#user-stories) - [Story 1](#story-1) - [Story 2](#story-2) - [Tradeoffs/Constraints/Caveats/Notes](#tradeoffsconstraintscaveatsnotes) - [References](#references) - [Alternatives (Optional)](#drawbacksalternatives) - [Workload Context](#workload-context) - [Test Plan](#test-plan) - [Scoring](#scoring) - [Implementation History](#implementation-history) ## **Release Signoff Checklist** Items marked with (R) are required. - [ ] (R) CBPP approvers have approved the CBPP status as `implementable` - [x] (R) CBPP summary, motivation and best practice details are appropriately documented - [ ] (R) Test plan is in place, giving consideration to CNF Test Suite input - [x] (R) Scoring has been determined - [x] "Implementation History" section is up-to-date - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes ## **Summary** Containers have a list of their own users independent of the host system, one of which is UID 0, the root user. Containers should run processes as a user other than root which makes it easier to run these images securely. Processes in a container should not run as root, nor should they assume they are running as root (UID 0). ## **Motivation** Both of these benefit the CNF developer directly, improving the quality of the CNF. Validation of the best practice is the responsibility of the CNF developer. Indirectly, improved CNF quality benefits CNF operators. The proposed tests are runnable by CNF operators as acceptance tests. SE-Linux based environments will require dropping root privileges. Example: OpenShift ### Goals Avoiding the root user in containers improves the security and behaviour of applications when bugs occur. It is also part of a defense in depth strategy against external compromises. Avoid compromised apps causing more damage. ### Non-Goals This BP recommends that the application not use the UID that can override all protections. This means that file read and write protections can be established. It does not consider what filesystem write permissions should be in order to benefit from those protections. CNF developers will additionally want to ensure filesystem permissions are tightened up appropriately so that non-root users are prevented from doing damage. This is outside the BP scope. ## Proposal When building a container, the container should be build to run its processes as a non-root user. setsid processes should not be required to do the work inside a container. The root user in a container has fewer rights and is distinct from the platform's root. The containerization process included using a user cgroup, which means that the containers' users are distinct from the platform's users and do not include most dangerous elevated rights. However, the container's root user does have full read/write access to the container's filesystem. It can read or modify any file. No secrets can be kept from it; it cannot be prevented from changing the content of all executable files on the system. On a basic level, avoiding the root user means that the container filesystem permissions are enforceable against all processes running in the system. Those processes can be prevented from doing critical things like: - viewing secrets they should not be viewing - modifying binaries within the filesystem that will later be executed Obviously, a well-written CNF would not be attempting to do things it should not do. But all software has bugs. Also, executing processes can be compromised by outside forces, and if this happens filesystem protection is a part of a "Defense in depth" strategy to ensure the compromise does not escalate. Unix access enforcement (user/group) will be respected. As an added advantage, fine-grained access enforcement, such as SELinux, will also hold. ### User Story _TBD: the user stories for access control need writing_ ### Tradeoffs/Constraints/Caveats/Notes Container images are frequently built from upstream image versions made from OS deployments. These will include things like setsid binaries as a part of their base configuration. If CNF developers follow this best practice they will have to audit and clean up any upstream images to respect this rule (removing files, removing packages or changing permissions as appropriate). By default, the first process starting in a container runs as root - you have to actively take steps to shed the permissions (Dockerfile USER line, plus installing files with appropriate ownership within the Dockerfile). We specifically want to run unprivileged process so that their access is limited. Of course, if access is limited then the CNF developer must ensure that access is still available to the things the CNF is going to need to access. This will involve changing permissions on data files and on working directories when the container image is constructed (they cannot be changed on startup because the application does not have the right to do that). This may also affect the use of shared volume mounts or host mounts - ownership of and rights on the root directory must permit access to the container users. ### References - CNF WG discussion: https://github.com/cncf/cnf-wg/discussions/20 - Sysdig: [Top 20 Dockerfile best practices](https://sysdig.com/blog/dockerfile-best-practices/) - 2021/03/09 - [Best practices for pod security in Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/developer-best-practices-pod-security) - 2020/07/28 - [RedHat OpenShift Runtime Security Best Practices](https://www.openshift.com/blog/openshift-runtime-security-best-practices) - K8s Documentation - Securing a Cluster: [Controlling what privileges containers run with](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#controlling-what-privileges-containers-run-with) - [Security Best Practices for Kubernetes Deployment](https://kubernetes.io/blog/2016/08/security-best-practices-kubernetes-deployment/) - 2016/08/31 - [Docker and Kubernetes — root vs. privileged](https://itnext.io/docker-and-kubernetes-root-vs-privileged-9d2a37453dec) - 2020/06/25 - Bitnami on [Use Non-Root Containers](https://docs.bitnami.com/kubernetes/faq/configuration/use-non-root/) - 2020/05/13 - [Running non-root containers on Openshift](https://engineering.bitnami.com/articles/running-non-root-containers-on-openshift.html) - K8s 11 Ways Not to Get Hacked: [8. Run Containers as a Non-Root User](https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/#8-run-containers-as-a-non-root-user) - Red Hat PDF [A PRACTICAL INTRODUCTION TO CONTAINER SECURITY](https://www.redhat.com/files/summit/session-assets/2017/L99901-apracticalintroductiontocontainersecurity_labguide.pdf) - 2017/05 ### Alternatives These are not strictly alternatives as they can be used with non-root, but can be applied to a container running as root. - Disable all capabilities or limit them - Do not run the container is privileged ## Workload Context All pod types should implement this best practice. ## Test Plan Inside a compliant application, all containers will avoid use of root. This CBPP will be tested by the CNF Test Suite. ### Static analysis A container image can be tested for compliance: - The container metadata should indicate that the first process started is started as a non-root user - The container filesystem will not have setsid-root binaries. If available, the Dockerfile can be checked to see if a non-root user is used. (Dockerfiles are not the only way to build containers, and the Dockerfile may not be a part of the CNF deliverable.) - See USER and RUN commands, both of which allow the Dockerfile author to express which user is to be used when launching the container. The above static analysis definitively confirms that a container cannot elevate privileges to local root as it removes all avenues for doing so. ### Runtime analysis One can check for processes launched as, or running as container root. We offer the following applications as examples that operators might wish to evaluate for this purpose: - Cnitch (https://github.com/nicholasjackson/cnitch) periodically checks the list of running containers in a Docker environment to see if any are running as container root. - Falco (https://falco.org/) checks for processes running as container root if the non-root container policy is set. Scanning systems that periodically check running processes or may not identify all root-owned processes, as it must conduct a scan at the moment a process is running. Similarly, process monitoring will not identify a problem if behaviour requiring a root process is not triggered. This cannot be used as a definitive guarantee of safety but is useful as a secondary check. ## Scoring This best practice results in a pass/fail on two counts, depending on role. Static analysis (all items checked for a pass) - CNF developers (testing before delivery) or CNF operators (testing what is delivered): - Container images indicate their processes should be started as a user other than 0 - Container images should not contain setsid-root binaries (user 0, u+s) Runtime analysis - CNF operators: - Operators may use run-time verification, from outside the application, to confirm that containers in processes are not owned by container root ## Implementation History First version: July 2021