# The joys of multi-cluster lifecycle management

2022-02-02 11am EST

Attendees:

- Kris Nóva (Twilio)
- Jay Pipes (AWS)

## Questions

1. How many clusters under management? 70
2. How big are the clusters?

3-200 nodes

5. What versions of Kubernetes are you running for those clusters? 1.19+
6. What kinds of workloads are they running (temporal? stateful? mixed?)? temporal now
7. Do you distribute ingress load for a single logical service across multiple clusters?
8. What addons or daemonsets are you running in them (and are those addons on similar/same versions)?
9. Are you using Helm or some other package manager for your k8s-native applications? Using Helm for infrastructure-y things?
10. Are you using GitOps practices? Using FluxCD or ArgoCD?
11. What are your top 3 pain points from an operator perspective? What, if any, workarounds or stop-gap measures have you implemented to get around those pain points?
- Nóva:
- Cluster drift (version, OS, components, etc)
- Networking and domains (accounts)
- Cluster management, identification, tracking, asset management, etc
- Upgrades
- Ross
- 70 clusters across several accounts
- Accounts vs domains vs teams requires a lot of work
- Tracking domain boundaries
- IAM and account limits

- Does EKS have a preferred life cycle manager?
- The reason we are all here.
- Why?
- How many customers are using cluster-api?
- Do they manage multiple accounts or have many management clusters?
- How do they setup the permission model for using cluster-api?
- How is EKS often deployed?
- Is it recommended to use a VPC for each cluster?
- Is eksctl frequently the tool of choice?
- How do we limit IP use and exhaustion?
- What is coming next for EKS?
- What is Amazon’s relationship with the cluster API provider?
- Is EKS going to fully support overlay networking in the future?
- Is EKS going to support changing/customizing the controlplane flags more?
- Is EKS going to support more automated upgrades (rather than node group by node group)?

---

- [ACK](https://github.com/aws-controllers-k8s/community)
- AWS Controllers for Kubernetes
- Jay: How are you running it today?
- Ross: We have an internal platform built on top of EKS
- Now at a point where its in debt, its limiting
- Autoscale groups were set up before EC2 ASG
- Mike
- New cluster template coming
- On the roadmap for this year :tada:
- EKSctl
- nope
- Cluster API
- they are investing
- Terraform
-
- Cloud Formation
-