## How to "unexpectedly" break a Kubernetes cluster? --- ## Three Principles - Default values everywhere. 只要能跑，預設就好。 - Two should be enough. 一個不夠，那就兩個。 - Client is always right. 客戶永遠是對的。 ---  --- ## Default values everywhere - Readiness && liveness probes - Resource requests & limits --- ## Demo: Readiness && liveness probes I am running a slow application --- ## Result: Readiness && liveness probes - Pod will never be ready. - Pod will restart forever. [deminy/delayed-http-response](https://registry.hub.docker.com/r/deminy/delayed-http-response) --- ## Demo: Resource requests & limits my application is a memory monster [code snippet](https://gist.github.com/tpai/d20cd63c6692d8f67fa46e6d29cbf4b9#file-delayed_http_response-yml) --- ## Result: Resource requests & limits - Consume as many resources as are available on the node - OOM Killed - Deployment keeps failing [dj80hd/resource-consumer](https://hub.docker.com/r/dj80hd/resource-consumer) [code snippet](https://gist.github.com/tpai/d20cd63c6692d8f67fa46e6d29cbf4b9#file-resource_consumer-yml) --- ## Two should be enough  --- ## Two should be enough for Kubernetes upgrade - Pod disruption budget --- ## Demo: Pod Disruption Budget#1 2 nodes 2 pods and maxUnavailable set to 0 or 0% --- ## Demo: Pod Disruption Budget#2 2 nodes 2 pods and minAvailable set to 2 or 100% --- ## Demo: Pod Disruption Budget#3 2 nodes 2 pods and pods are in the same node using node affinity or taint --- ## Result: Pod bisruption budget No node were upgraded. --- ## Client is always right  --- ## Demo: HTTPS certificate Client sent me a SSL cert without intermediate and root. Use [this handy tool](https://www.nicsrs.com/download-cert-chain) to download full chain certification. --- ## Result: HTTPS certificate - Certificate is not trusted in all web browsers. - Certificate won't work in legacy system. --- ## Recap - Default values everywhere. 只要能跑，預設就好。 - Two should be enough. 一個不夠，那就兩個。 - Client is always right. 客戶永遠是對的。 --- ## Takeaways - Read document! 看文件！ - Read document! 看文件！ - Don't trust your client. 不要相信客戶。 --- ## Quotes --- > Hope pain and suffering happens to you. > 成功需要很多的苦難，願你們都能經歷痛苦和磨難。 > --Jensen Huang --- > Hope incident and downtime happens to you. > SRE 的成長需要很多經驗，願你們都能經歷事故和斷線。 > --Tony Pai