# Background, References, and Ideas for potential CNF Best practices

## Background context

Goal for CNF best practices:
> Determine what practices will allow telco applications to most efficiently utilize K8s to lower the operational risk and burdens of SPs and telco vendors

Production Telecom K8s environment:
> k8s cluster based on upstream k8s + CNI + CSI backed by adequate persistant storage + Container Registry + Helm repository and most likely + some standard observability and logging add-ons + Git and some CICD pipeline connected to it. The CNF is free bring its own (cloud native) add-ons and install them of this platform.
- ref: https://cloud-native.slack.com/archives/C01F1LVAQCC/p1606991123008800

Types of best practices
- Life cycle management of a CNF
- Audience:
- primary: SP ops team (internal or outsourced)
- secondary: vendor support team
- deployment to cloud infrastructure
- operation management including automation from orchestrator
- Consumption of cloud resources and capabilities (services?)
- Audience:
- primary: CNF creator
- secondary: SP

Are we helping to save on CAPEX or OPEX?

Are we helping to reduce integration pains with multi-vendor CNFs? Ideally it is zero touch automation. Interoperability. There are similarities between a CNF and the platform.

Guiding criteria for choosing a proposal to focus on:
- Can you express the idea, requirement, or test as a best practice
- Not specific functional requirements
- Is there content that can be used and referenced
- Is this a best practice that can be discussed, debated, and measured

## References:
- [K8s Configuration Best Practices
](https://kubernetes.io/docs/concepts/configuration/overview/)
- https://cloud.google.com/blog/products/containers-kubernetes/tips-and-tricks-for-developers-learning-to-work-with-gke
- [Google Cloud Best Practices](https://cloud.google.com/docs#section-2)
- [14 best practices for developing on OpenShift](https://www.openshift.com/blog/14-best-practices-for-developing-applications-on-openshift)
- https://resource.alibabacloud.com/bestpractice/index.htm
- https://cloud.google.com/solutions/twelve-factor-app-development-on-gcp
- https://medium.com/ibm-cloud/kubernetes-12-factor-apps-555a9a308caf

## Ideas:

1. * Container does not require local persistent storage - A container may use local storage if available, but must not rely on it for persistent storage of data
> [name=Denver] :+1: good, but contentious
> [name=Drew] :+1: I like but yes, contentious, lots of what if followups
1. * K8s Workload resource (Pod, Deployment, ReplicaSet) works as expected after being rescheduled to a new node
> [name=Denver] :thumbsup: lots of material on this
> [name=Taylor] what is the best practice?
1. Container should run unprivileged
> References:
> https://x.cnf.dev/process-containers/
> [Oct 2020 cncf blog post](https://www.cncf.io/blog/2020/10/16/hack-my-mis-configured-kubernetes-privileged-pods/)
3. CCDP-002 container is responsive after reset
> See PR #7 [Create 0002-container-is-responsive-after-a-reset](https://github.com/cncf/cnf-wg/pull/7)
5. CNF will run on any certified K8s
> [name=Drew] :+1: This could be a good start on a proposal, seems more straight forward.
> [name=Taylor] :+1: CNF WG members are already mentioning this.
7. Reasonable image size of each container within a K8s Workload Resource
> [name=Drew] Another one I like but could be very contentious; decent info found at [Kubernetes best practices: How and why to build small container images](https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-how-and-why-to-build-small-container-images) when researching best practice on image size for containers.
> _"performance and security advantages"_
9. CNF / Workload and/or Workload resources should support autoscaling by orchestration (see horizontal-pod-autoscale)
10. Don’t hardcode IP addresses or subnet masks in the K8s runtime configuration  (non-declarative)
11. Don’t specify a hostPort for a Pod unless it is absolutely necessary.  (see k8s best practices for #services)
12. Avoid using hostNetwork, for the same reasons as hostPort.
13. CNF configuration should be immutable after deployment
* CNF == K8s Workload
> [name=Denver] :+1: good initial best practice proposal
* Ref:
* https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable
* https://kubernetes.io/docs/concepts/configuration/configmap/#configmap-immutable
14. K8s Workload Resource configuration should be immutable after deployment
* K8s Workload Resource == CNF Components
* Ref:
* https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable
* https://kubernetes.io/docs/concepts/configuration/configmap/#configmap-immutable
15. CNF should not directly access host hardware resources (e.g. accessing the host /dev or /proc from a mount)
> [name=Denver] :+1: good and less contentious. Operators could be used. CNF containers should not access directly.
> [name=Taylor] How can we write this as a general best practice.
16. CNF should be tolerant / resilient to containers failing/being killed
17. Workload Resources should be tolerant / resilient to containers failing/being killed
18. Container provides API for health status monitoring
> see PR #8 [Create 0003-container-provides-api-for-health-status-monitoring](https://github.com/cncf/cnf-wg/pull/8)
19. Keep application configuration outside of the image
- eg.
> Keep the environment-specific configuration outside of the container image. For example, use ConfigMaps and Secrets to store the application configuration.
21. Container should execute process as non-root user
* SE-Linux based environments will require dropping root privileges. Example: OpenShift
* Potential test: look at Dockerfile for user mapping
* Ref:
- https://engineering.bitnami.com/articles/running-non-root-containers-on-openshift.html
- https://docs.openshift.com/container-platform/3.3/creating_images/guidelines.html
- https://docs.openshift.com/enterprise/3.2/admin_guide/manage_scc.html