Try   HackMD

Office Hours for 16 Sept

West Edition

Panelists

Jorge Castro Derrik Campau Vamshi Samudrala Josh Berkus

Question: Hi. I've set up a multi master cluster with kubeadm. Everything is working fine but I want to make some changes to the arguments passed to the api-server. What is the best way to do that? Some suggest changing a config file, some talk about a config map, some mention kubeadm config and some talk about /etc/kubernetes/manifests. What is the prefered way to add arguments to that and do I need to do that on each master individually?

kubeadm upgrade plan - writes up to a config file kubeadm config view

https://kubernetes.io/docs/tasks/run-application/configure-pdb/ https://kubernetes.io/docs/concepts/workloads/pods/disruptions/

Question: how does ingress resource map to ingress service/pods? I see kubernetes.io/ingress.class=nginx but not sure how this translates/links to nginx-ingress-controller service?

https://kubernetes.github.io/ingress-nginx/user-guide/multiple-ingress/ https://docs.nginx.com/nginx-ingress-controller/installation/running-multiple-ingress-controllers/ https://kubernetes.github.io/ingress-nginx/user-guide/basic-usage/

Question: Has anyone ever encountered master nodes (ec2, c5.24xl, k8s 1.15.9) randomly lock up, preventing etcd cluster from achieving quorum? By locking up I mean that node becomes inaccessible via SSH and ec2 metrics show CPU usage drop to 0. I'm seeing this after hitting a large number of objects in etcd.

Question: Hello, does anyone know how to consume all pod failure events? I want to see things like probe failures but also I want to see failures in the Horizontal Pod Autoscaler. I was looking at the audit log webhook but I’m not sure if this has this information.

Question: Can anyone help me? I am lost. I have tested an application on my local and it works fine. However, when I deploy using kubernetes, the pods seem like they’re working fine, but I cannot get the ingress to work. I’ve seen various issues, but now my cluster has seem to have gotten into a state where I cannot install the ingress anymore. Can anyone walk me through setting up an ingress like I’m 5??

https://kubernetes.github.io/ingress-nginx/deploy/ https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/ https://kubernetes.io/blog/2020/08/21/moving-forward-from-beta/

Question: Does anyone know if it's possible to get the latest resourceVersion number from the cluster? I ask because I want to start a bunch of watchers without replying the current state snapshot (when you set resourceVersion: null).

Question: we're going to have to stand up a new DNS server because external-dns doesn't work well with msft AD – has anyone had much experience in this area? Looking at suggestions on what to use for the server – I would think BIND or CoreDNS would be the logical choices, but would like more opimions.

EU Edition

Panelists

Jorge Castro Pierre Humberdroz Rachel Leekin Povilas Versockas

Thanks to the following companies for lending us these experts: Google, Spectrm, Microsoft, VMware, StockX, Giant Swarm, UW

Questions:

Person: doogie

Question: Got a question for you. My requirement is dependable egress static IP’s for my cluster, so that my vendors can whitelist our access. Ideally we’d be able to segment the namespaces or pods to specific node groups so that we could have blocks of IP’s for different purposes. Running on GKE, it looks like one of our only options, if not the only option, is to use a private cluster and Cloud NAT which has its own host of issues. What’s the ideal/recommended solution for this?

Urls:

Person: Simone Baracchi

Question: A question about DNS. Can I use different DNSes in different pods? For example I want pod 1 to use a certain MS Active Directory DNS service while pod 2 to use another. I know I can overwrite /etc/resolv.conf, but I can only use one DNS server, so I can either use the MS-AD one, or the kubernetes one, but not both, so my pod will lose ability to resolve the k8s internal URLs. Or I can set it globally but all pods will have the same DNS server. Can I have pods using a specific one?

Urls:

Person: Evesy

We’re using an init container (centos 7) to do some iptables rules to redirect traffic to localhost:6379 to $REMOTE_IP:6379 which works fine. We’ve been moving to centos 8 containers for our base now, which replaces iptables with nftables, I’ve got the equivalent nft commands however I’m always getting: Error: Could not process rule: Operation not supported The init container is running with NET_ADMIN (as it was before) and I’ve also tried adding NET_RAW. It works successfully just running via Docker on my local machine with the same capabilities using –cap-add but I can’t get it working in Kubernetes. Could this be down to the underlying OS on the k8s nodes (GKE CoS)? Or some other factor I haven’t considered?

person: Christian Roy

Question: Did anyone on the panel ever migrated from GKE to EKS and what were the things that didn't quite work the same way or what possible issues would you say I should look for?

person: Angelos Mimidis

Question: Any tool recommendations for kubernetes security bench marking? Everything I have seen so far, is not configurable enough to cater for "non-vanilla" deployments of k8s.

urls: https://github.com/bridgecrewio/checkov https://github.com/aquasecurity/kube-hunter

person: Yogi Question: Contributing to k8s documentation is very complicated. Is there a video tutorial? Create issues, create pr,

Person: Agustín Houlgrave Question: I've seen good things in argo, for example argo's custom rollout, which is a deployment but nicer. The fact that this exists i could mean that vanilla-k8s' Deployment may be getting a little bit incomplete. do you know if there are any plans for improving the Deployment resource?

person: Yogi Is podpreset is deprecated? What would be a good replacement if so?

Intro Script

Welcome everyone to today’s Kubernetes Office Hours, where we answer your user questions live on the air with our esteemed panel of experts. You can find us in [#office-hours] on slack, and check the topic for the URL for the information.

  • Before we begin let’s start by introducing ourselves: (Give each panelist about a minute)
  • Before we start here are the ground rules:
    • This is a Kubernetes event so the Code of Conduct is in effect, please be excellent to each other.
    • This is a judgement-free zone, everyone had to start from somewhere so please help out your buddy by having a supportive environment in the channel.
    • While we will do our best to answer your questions the panel doesn’t have access to your cluster, so live debugging is off topic, but we will do our best to get you moving down the next step.
    • Panelists, you’re encouraged to expand on answers with your experiences and pro-tips.
    • Audience, you can help by pasting in URLs to official docs, blogs, or anything that might be relevant to the topic at hand.
    • Post your questions on [discuss.kubernetes.io].
    • You can also help us out by tweeting, spreading the word, and paying it forward.
    • This panel is made entirely of volunteers, if you want to rotate in please let us know, we love to have new people rotate in and help out.

Contest

The hack.md notes document will have a list of who has asked questions, roll a dice to see who won the shirts. On occasion if someone from the audience has been helpful feel free to give them a shirt as well, we want to reward people for helping others. Note: Multi-sided dice not included.

Outro

(Note, the companies will change over time depending on the hosts)

  • Thanks to the following companies for supporting the community with developer volunteers: Giant Swarm, StockX, Pivotal, Pusher.com, Weaveworks, VMware, University of Michigan, Red Hat, and Utility Warehouse. Spectrm.

    Special thanks to CNCF for sponsoring the t-shirt giveaway.

And lastly, feel free to hang out in [#office-hours] afterwards, if the other channels are too busy for you and you’re looking for a friendly home, you’re more than welcome to pull up a chair and hang out.

Last changed by 
 
Kubernetes
Production grade container orchestration, made with love
0
372

Read more

Lessons Learned from 3 years of Kubernetes Office Hours

Date: 23 August 2020 Authors: Jorge Castro (VMware), Josh Berkus (Red Hat), Chris Carty (Google), Dan "POP" Papandrea (Sysdig), David McKay (Equinix Metal) Today we celebrate three years of the Kubernetes Office Hours. This is a monthly event where we take a panel of volunteers, stick them on a live stream, and then see how many questions we can field from the community. I started the show for one reason. I had recently started my Kubernetes journey and was learning all these new concepts while rewiring my traditional sysadmin brain to be more cloud native. So the idea was if I'm going to dig into this stuff and bother my new coworkers with silly questions, we might as well do it together and on the air to share our experiences and do it in a way that is fun and useful for others. Give away some tshirts, fame and fortune would surely follow. After 65 episodes we've decided to take a look at some of the more common problem areas that we've been tackling, and put together a quick summary for you on things where you might want to invest your attention. You will find many articles on "Top X things to know about Kubernetes". We've specifically avoided those and went back into our archives because what people think you need to know and what you actually need to know can be different.

Mar 18, 2021
Office Hours Feb 2021

EU Edition Panelists Person: Andrew Question: We're writing a controller with controller-runtime, and trying to use the Generation/ObservedGeneration pattern to avoid reconciling if there isn't any change (not using the predicate provided by controller-runtime for that purpose yet though). My question is how can that work with the possibility of a stale cache? When we write the ObservedGeneration to the Status of our CR, it triggers another reconcile immediately, but in some cases, the cache is stale and the CR it "Get"s still has the old Status, and therefore the old ObservedGeneration. What is the recommended strategy of dealing with this? Thanks! Person: Simone Baracchi Question: I'd like to configure my small cluster as "highly available" with no single master / single point of failure and make the best use of all the cluster resources. My current plan is to make 3 nodes run as masters and be able to schedule pods on the masters. From my research the issues in doing so are 1) security issues about sensitive data on master which could be read from malicious pods and 2) pods competing for resources (especially in case of a node failure). I'm not too concerned about security atm, and I can think of limiting the max number of pods / resources used. Is there any other red flag in doing so? Person: Jesper Berg Axelsen

Feb 17, 2021
Contributors Awards Ceremony Playbook

awards/ README.md -- list of everyone playbook.md Kubernetes has a tradition of yearly peer awards. These are commonly handed out during the end-of-year Contributor Summit. The CNCF typically has awards as part of KubeCon + CloudNativeCon, the Kubernetes awards are a compliment to those, and are meant to be peer awards. Ideals and Values Typically each SIG will nominate two(2) recipients for an award.

Feb 12, 2021
Office Hours January 2021

EU Edition Panelists Rachel Leekin, Chris Carty, Dan POP Papandrea, Saiyam Pathak Person: Mostafa Elmenbawy (https://kubernetes.slack.com/archives/C6RFQ3T5H/p1609991530274100?thread_ts=1607960423.257700&cid=C6RFQ3T5H) Question: What is recommended for on premise production cluster spanning multiple hosts? Answer:

Jan 20, 2021
Read more from Kubernetes
Published on HackMD