tags: Meetup、Co-writing

CNTUG Meetup #20

Talk 1: End to End Testing for Kubernetes Cluster

Kubernetes testing

1. why do I need to test cluster?

• 分離 cluster or application 的問題
• 在上production之前確保Cluster是正常的

2. testing types

   • unit testing
   • integration testing
   • e2e testing: 模擬使用者角度

3. Kubernetes SIGs -
   SIG Testing => test infra

4. E2E Testing - kubetest

   • test-infra provides kubetest for K8s cluster testing

   • notice: -provider and –deployment 需一致, 否則kubetest會報錯

   • kubetest flow:

     • hack/e2e.go -> test-infra/kubetest -> hack/ginko-e2e.sh -> tes/e2e/e2e_test.go
       (會自己 create 自己 e2e 的 namespace，結束後會自己刪掉)

e2e.go 會先找環境中有沒有kubetest，沒有的話就會去下載，kubetest下來，如果有的話，就會更新kubetest。

Ginkgo & Gomega

BDD: 以軟體行為來描述測試，讓非技術人員也能了解。

Ginkgo
is a Behavior-Driven Development (BDD) style Go testing framework

Ginko-e2e

Kind of test

• slow
• serial
• Disruptive
• Internet
• Feature:..
• Conformance
• LinuxOnly
• Privilage
• …

Writing Test

1. Debuggability
2. Ability to run in non-dedicated test cluster
3. Speed of execution
4. Resilience to relatively rarw, temporary infra glitches or delays

GitHub Sample

https://github.com/pohsienshih/kubernetes-e2e-practice

Testing tools

Sonobuoy

CNCF certified K8s test tool

• Pros: easy
• Cons: Not guarantee to get the newest version of K8s

Node test

測試kubelet能否正常的執行在node上面。

• Only support Linux host.
• Only support local and GCE deployment.

執行方式：

1. kubetest (只支援GCE)
2. make test-e2e-node

Conclusion

• E2E Testing 可以減少定位的成本
• 定期執行 e2e testing

Q&A

Q1: Ginkgo 是否可以用在自己的服務（非K8s）
A1: Ginkgo是可以用在一般服務的，因為他是golang BDD-testing framework，只是K8s拿來使用，但是用Ginkgo要注意dump result的格式，

權限問題 => E2E會去執行label[Provileged]測項，這個會需要比較大的權限，而Conformace Testing不需要很大的權限， e2e 才需要， 詳情可參見 kind of tests - Privileged

---

Talk 2: 不會 Go 語言也能寫 K8s Controller 之 Metacontroller

• What is Controller & Operator Pattern
• Build Operator with client-go

Operators vs. controller pattern

How K8s Works

1. create deployment 流程
   api-server -> 存deployment資訊到 etcd -> deployment controller create pod record到 etcd -> scheduler 會去 etcd 看有哪些 pod 沒有分配到 node ，然後指派 node 給他 -> 被綁到的那個 node 上的 kubelete 會實際上去部署那個 pod。

2. Operator Pattern = (Custom Resource Definition)CRD + Custom Controller
   Why => deploy 之後只能仰賴K8s的機制去處理, operator可以處理Deploy之後的細節

metacontroller 是把底層細節包裝起來

Controller

• informers - get information for etcd
• workqueue - except from thread safe, work queue limit object id in one set, two workers are forbidden to access same object.
• control loop - goroutines as looper

Workqueue

• Client-go提供informer work queue library
• Code-generator
• apimachinary

Metacontroller

Google開發的把Operator需要的package都包在一起。
要做的只有兩件事：

1. CRD
2. Management logic as webhook

用Code的方式去執行kubectl apply等指令

---

Talk 3: 分享如何在 GitHub 上加入 Kubernetes 組織

Organizations

• Kubernetes
• Kubernetes SIGs
• Kubernetes Incubator

sig-testing/test-infra

Prow

1. OWNERS - Defined reviewers and approvers
2. Auto Assigned revierers
3. Some plugins that you can use
4. Merged by comments (lgtm, approved)

補充：Prow的運作流程

CI: Drone / CD: Skaffold

Tide

1. Define merge methods
2. List of labels and giving PR

Apply to join kubernetes organization

• Kubernetes/org
• Prerequistite:
  1. sponsored by 2 reviewers or approvers.
  2. multiple contributions to tnhe project.
  3. enabled two-factor authentication on their GitHub account.
  4. subscribed to [e-mail].

Tips:

• Due to long time for reviewers to review the PR, you might push your PR reviewing process by notifying on SLACK.
• 人多的地方不要走（Ｘ

Q&A

Q: What is the contributing flow?
A:

Lightning Talk

小飛機與你有約(蕉流時間)