--- # System prepended metadata title: 'Background, References, and Ideas for potential CNF Best practices' --- # Background, References, and Ideas for potential CNF Best practices ## Background context Goal for CNF best practices: > Determine what practices will allow telco applications to most efficiently utilize K8s to lower the operational risk and burdens of SPs and telco vendors Production Telecom K8s environment: > k8s cluster based on upstream k8s + CNI + CSI backed by adequate persistant storage + Container Registry + Helm repository and most likely + some standard observability and logging add-ons + Git and some CICD pipeline connected to it. The CNF is free bring its own (cloud native) add-ons and install them of this platform. - ref: https://cloud-native.slack.com/archives/C01F1LVAQCC/p1606991123008800 Types of best practices - Life cycle management of a CNF - Audience: - primary: SP ops team (internal or outsourced) - secondary: vendor support team - deployment to cloud infrastructure - operation management including automation from orchestrator - Consumption of cloud resources and capabilities (services?) - Audience: - primary: CNF creator - secondary: SP Are we helping to save on CAPEX or OPEX? Are we helping to reduce integration pains with multi-vendor CNFs? Ideally it is zero touch automation. Interoperability. There are similarities between a CNF and the platform. Guiding criteria for choosing a proposal to focus on: - Can you express the idea, requirement, or test as a best practice - Not specific functional requirements - Is there content that can be used and referenced - Is this a best practice that can be discussed, debated, and measured ## References: - [K8s Configuration Best Practices ](https://kubernetes.io/docs/concepts/configuration/overview/) - https://cloud.google.com/blog/products/containers-kubernetes/tips-and-tricks-for-developers-learning-to-work-with-gke - [Google Cloud Best Practices](https://cloud.google.com/docs#section-2) - [14 best practices for developing on OpenShift](https://www.openshift.com/blog/14-best-practices-for-developing-applications-on-openshift) - https://resource.alibabacloud.com/bestpractice/index.htm - https://cloud.google.com/solutions/twelve-factor-app-development-on-gcp - https://medium.com/ibm-cloud/kubernetes-12-factor-apps-555a9a308caf ## Ideas: 1. * Container does not require local persistent storage - A container may use local storage if available, but must not rely on it for persistent storage of data > [name=Denver] :+1: good, but contentious > [name=Drew] :+1: I like but yes, contentious, lots of what if followups 1. * K8s Workload resource (Pod, Deployment, ReplicaSet) works as expected after being rescheduled to a new node > [name=Denver] :thumbsup: lots of material on this > [name=Taylor] what is the best practice? 1. Container should run unprivileged > References: > https://x.cnf.dev/process-containers/ > [Oct 2020 cncf blog post](https://www.cncf.io/blog/2020/10/16/hack-my-mis-configured-kubernetes-privileged-pods/) 3. CCDP-002 container is responsive after reset > See PR #7 [Create 0002-container-is-responsive-after-a-reset](https://github.com/cncf/cnf-wg/pull/7) 5. CNF will run on any certified K8s > [name=Drew] :+1: This could be a good start on a proposal, seems more straight forward. > [name=Taylor] :+1: CNF WG members are already mentioning this. 7. Reasonable image size of each container within a K8s Workload Resource > [name=Drew] Another one I like but could be very contentious; decent info found at [Kubernetes best practices: How and why to build small container images](https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-how-and-why-to-build-small-container-images) when researching best practice on image size for containers. > _"performance and security advantages"_ 9. CNF / Workload and/or Workload resources should support autoscaling by orchestration (see horizontal-pod-autoscale) 10. Don’t hardcode IP addresses or subnet masks in the K8s runtime configuration  (non-declarative) 11. Don’t specify a hostPort for a Pod unless it is absolutely necessary.  (see k8s best practices for #services) 12. Avoid using hostNetwork, for the same reasons as hostPort. 13. CNF configuration should be immutable after deployment * CNF == K8s Workload > [name=Denver] :+1: good initial best practice proposal * Ref: * https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable * https://kubernetes.io/docs/concepts/configuration/configmap/#configmap-immutable 14. K8s Workload Resource configuration should be immutable after deployment * K8s Workload Resource == CNF Components * Ref: * https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable * https://kubernetes.io/docs/concepts/configuration/configmap/#configmap-immutable 15. CNF should not directly access host hardware resources (e.g. accessing the host /dev or /proc from a mount) > [name=Denver] :+1: good and less contentious. Operators could be used. CNF containers should not access directly. > [name=Taylor] How can we write this as a general best practice. 16. CNF should be tolerant / resilient to containers failing/being killed 17. Workload Resources should be tolerant / resilient to containers failing/being killed 18. Container provides API for health status monitoring > see PR #8 [Create 0003-container-provides-api-for-health-status-monitoring](https://github.com/cncf/cnf-wg/pull/8) 19. Keep application configuration outside of the image - eg. > Keep the environment-specific configuration outside of the container image. For example, use ConfigMaps and Secrets to store the application configuration. 21. Container should execute process as non-root user * SE-Linux based environments will require dropping root privileges. Example: OpenShift * Potential test: look at Dockerfile for user mapping * Ref: - https://engineering.bitnami.com/articles/running-non-root-containers-on-openshift.html - https://docs.openshift.com/container-platform/3.3/creating_images/guidelines.html - https://docs.openshift.com/enterprise/3.2/admin_guide/manage_scc.html