# Office Hours Feb 2021 ## EU Edition ### Panelists --- Person: Andrew Question: We're writing a controller with controller-runtime, and trying to use the Generation/ObservedGeneration pattern to avoid reconciling if there isn't any change (not using the predicate provided by controller-runtime for that purpose yet though). My question is how can that work with the possibility of a stale cache? When we write the ObservedGeneration to the Status of our CR, it triggers another reconcile immediately, but in some cases, the cache is stale and the CR it "Get"s still has the old Status, and therefore the old ObservedGeneration. What is the recommended strategy of dealing with this? Thanks! --- Person: Simone Baracchi Question: I'd like to configure my small cluster as "highly available" with no single master / single point of failure and make the best use of all the cluster resources. My current plan is to make 3 nodes run as masters and be able to schedule pods on the masters. From my research the issues in doing so are 1) security issues about sensitive data on master which could be read from malicious pods and 2) pods competing for resources (especially in case of a node failure). I'm not too concerned about security atm, and I can think of limiting the max number of pods / resources used. Is there any other red flag in doing so? --- Person: Jesper Berg Axelsen Question: Is it possible to limit a service account to only have rights to create, edit and delete custom resource definitions that are related to a certain namespace? Since CRDs are not namespaced, I only see the option to give my service account rights to create, edit and delete all CRDs on the cluster. For the system we are creating, we do not see this as secure and would like to know if there is a way to limit our service account? --- Person: knabben Question: I’m planning to install falco on our AWS nodes, what’s the best way to bring them up when using auto-scaling or adding new nodes to the pool. --- Person: Ram Iyengar Question: What are the general areas that K8s security is working on? - https://github.com/kubernetes/community/blob/master/sig-security/README.md - https://github.com/kubernetes/community/blob/master/sig-security/security-audit-2021/RFP.md - https://neonmirrors.net/post/2021-02/kubernetes-policy-comparison-opa-gatekeeper-vs-kyverno/ CKS Info - https://github.com/knabben/k8s-labs/blob/main/CKS/README.md - https://twitter.com/SaiyamPathak/status/1354102624836968449?s=20 - https://www.youtube.com/watch?v=NSAyBFZzkyk&feature=youtu.be - https://github.com/walidshaari/Certified-Kubernetes-Security-Specialist --- Person: Ankit Question: When I delete pvs (retention policy is Retain), pv remains in the system which is expected but when I delete that pv, I expect the underlying Volume to be deleted on The Cloud provider but that doesn't happen. So does this mean that one has to manually go and cleanup all the volumes which remains in the cloud due to this behaviour. One has to take care of this otherwise it will increase the unnecessary cost. I wish to create a feature request to enable the deletion of underlying volume on deletion of pv. What are your views on this. Am I missing something here ? I'm on EKS. --- Person: hfwun Question: I am completely new to this and am investigating whether it is worthwhile for my company to move to Kubernetes. We run several (smaller) websites and some of them get a lot of traffic after a social media post. My main concern is abstracting the manual provisioning of VM's because it is burdensome to document the configuration. I think it would be easier with containers. Any general advice on when it's worth moving to k8s? - https://www.giantswarm.io/on-demand-webinar-how-to-pitch-kubernetes-to-management - https://keptn.sh/ - https://nobl9.com/ - https://keda.sh/ --- Person: Bala Question: Hi are there any tools for finding and deleting dangled RBAC rules in kubernetes mainly onprem Doesn't look like there's any. :( IaC would help here Other tools to look at for RBAC reviews: - https://github.com/corneliusweig/rakkess - https://github.com/FairwindsOps/rbac-lookup Person: Mostafa Elmenbawy How can I use a GPU by multiple pods? i.e request fraction of a GPU from a pod - https://github.com/AliyunContainerService/gpushare-device-plugin?spm=a2c65.11461447.0.0.5d8b51ba4FKKKK&file=gpushare-device-plugin.git - https://github.com/AliyunContainerService/gpushare-scheduler-extender --- Person: awooolfgang Link: https://discuss.kubernetes.io/t/container-startup-sequence/14834 Question: Hello, i would like to ask if containers are started in the order they are written in a statefulset manifest file? “Then the kubelet runs the Pod’s init containers in the order they appear in the Pod’s spec.” (Init Containers | Kubernetes) Is this true for non-init containers? Answer: --- Person: metadbsd Link: https://discuss.kubernetes.io/t/there-a-way-to-validate-my-yaml-on-kubernetes-1-20/14824 Question: There a way to identify which part of my yaml is not compatible with Kubernetes 1.20 ? --- From twitter Person: @meetmeat05 Question: Is there any tutorial where i can learn Kubernetes from scratch? Answer: There are many options. - Kubernetes the hard-way https://github.com/kelseyhightower/kubernetes-the-hard-way - https://kubernetes.io/docs/tutorials/ - https://kubernetes.io/docs/concepts - https://github.com/ramitsurana/awesome-kubernetes - https://kube.academy/ - https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/ - https://kubernetesreadme.com/ - https://katacoda.com/kubernetes - https://www.udacity.com/course/scalable-microservices-with-kubernetes--ud615 - https://training.linuxfoundation.org/training/introduction-to-kubernetes/ - https://www.manning.com/books/kubernetes-in-action - https://www.udacity.com/course/scalable-microservices-with-kubernetes--ud615 - Play with KIND: https://kind.sigs.k8s.io/ - https://www.youtube.com/watch?v=X48VuDVv0do - https://www.redhat.com/cms/managed-files/cm-oreilly-kubernetes-patterns-ebook-f19824-201910-en.pdf - Ellen Korbes' tooling talk: https://www.youtube.com/watch?v=b1RsNXGLuUk - https://learning.oreilly.com/library/view/production-kubernetes/9781492092292/ Person: gowtham Answer: How to manage dns i.e., create, destroy sub domains etc… in route53 for public facing applications running on eks? - https://github.com/kubernetes-sigs/external-dns - Yogi: I use a simpler technique. Create an ingress controller. Gives me an LB Address. Create 2 route 53 entries. *.cluster.domain.com > IP cluster.domain.com > IP - Brian Davis: kube2iam to use annotations for isolated roles - https://github.com/open-policy-agent/gatekeeper-library/tree/master/library/general/uniqueingresshost - https://github.com/open-policy-agent/gatekeeper-library - https://github.com/open-policy-agent/library/tree/master/kubernetes - https://play.openpolicyagent.org/ - https://www.conftest.dev/#usage Mutation Notes: - https://kyverno.io/ - https://didil.medium.com/building-a-kubernetes-mutating-admission-webhook-7e48729523ed - Stream this week! https://www.youtube.com/watch?v=GlqCW7uJ-7Q - OPA Mutations: https://www.youtube.com/watch?v=zVuM7F_BTyc Person: Ruaridh Angus Question: Multi-Cluster service discovery between EKS and Bare Metal, currently running Consul but we have found some undesirable behaviour with health checking while using catalogue sync. Do we have a "preferred" / native solutions that are simple(well no more complicated than consul) and don't involve using headless services and then loosing load balancing like coreDNS kubernetai. # Appendix ### Intro Script Welcome everyone to today’s Kubernetes Office Hours, where we answer your user questions live on the air with our esteemed panel of experts. You can find us in [#office-hours] on slack, and check the topic for the URL for the information. - Before we begin let’s start by introducing ourselves: (Give each panelist about a minute) - Before we start here are the ground rules: - This is a Kubernetes event so the Code of Conduct is in effect, please be excellent to each other. - This is a judgement-free zone, everyone had to start from somewhere so please help out your buddy by having a supportive environment in the channel. - While we will do our best to answer your questions the panel doesn’t have access to your cluster, so live debugging is off topic, but we will do our best to get you moving down the next step. - Normally we provide shirts however the CNCF store is replenishing its inventory, we will give you a shoutout and our undying devotion - Panelists, you’re encouraged to expand on answers with your experiences and pro-tips. - Audience, you can help by pasting in URLs to official docs, blogs, or anything that might be relevant to the topic at hand. - Post your questions on [discuss.kubernetes.io]. - You can also help us out by tweeting, spreading the word, and paying it forward. - This panel is made entirely of volunteers, if you want to rotate in please let us know, we love to have new people rotate in and help out. ### Contest The hack.md notes document will have a list of who has asked questions, roll a dice to see who won the shirts. On occasion if someone from the audience has been helpful feel free to give them a shirt as well, we want to reward people for helping others. Note: Multi-sided dice not included. ### Outro (Note, the companies will change over time depending on the hosts) - Thanks to the following companies for supporting the community with developer volunteers: Giant Swarm, Sysdig, Phase2, Weaveworks, VMware, Red Hat, Equinix, Google, Microsoft, and Utility Warehouse. Special thanks to CNCF for sponsoring the t-shirt giveaway. And lastly, feel free to hang out in [#office-hours] afterwards, if the other channels are too busy for you and you’re looking for a friendly home, you’re more than welcome to pull up a chair and hang out.