# Client-go and API extensions @munnerz ## Goals: ### What is painful when building a controller #### Questions around best practices ##### After this week: - Someone new - What is particularly hard to grasp - Someone experienced - What important bits of info are critical? ##### Pain points when building controller - Boilerplate - work queues - HasSynced functions - Re-queing - Lack of deep documentation - some documentation but focused on k/k core - Securing webhooks & APIServices - Validation schemas - Webhooks/secuity is particularly difficult right now - openshift solution was mentioned - Scaling API is pretty low level - Registering CRDs, building and consuming custom controllers - Do I deploy the CRD as a deployment or have controller register the CRD? - building/useability is unclear - declarative API needs to translate to transactional APIs sometimes but there is no best practice/library. Ex. Ingress, specify ingress object - kubernetes resource model has API types separated - interaction with third party systems is possible but complicated and needs to be built - if third party system can't react to change there is no obvious solution. webhooks? not possible for asynchonous - controllers consuming kubernetes resources that interact with external systems are tricky ##### Best practices - questions - How many times to update status per sync - Conditions vs fields - used to check a pods' state. - idea behind conditions: you can add conditions that are ok for new clients to pay attention to and old clients to ignore - Validating webhooks vs OpenAPI schema - How are people testing extensions? - anyone resuing upstream dind cluster? - how else are people testing? - @pwittrock: kubebuilder: package control plane binaries, generate tests for resources and controllers - integration tests -- pull in packages that will build the clusters - kube adm to build single node cluster and running tests against that - travis ci - spin up new vms - Lee from Kubeadm - runs kubeadm on docker and docker node, does multinode and hopes to expand to multicluster: look up if you want to run it against dnd node - #testing-commons channel on stack. 27 page document on this--link will be put in slides - Deploying and managing Validating/Mutating webhooks? - how complex should they be? - When to use subresources - Are people switching to api agg to use this today? - Why not use subresources today with scale? - multiple replicas fields - doesn't fit polymorphic structure that exists - Phil Wittrock: kubectl side, scale ##### Moving forward - How can we best disseminate this knowledge - most sigs maintain controllers - wiki? developer docs WG? - existing docs focus on in-tree development - @pwittrock: [Working on a book](book.kubebuilder.io) for Kubebuilder and would appreciate feedback - Best place to have real-time conversation about these topics is: #sig-apimachinery - API extension authors meetings/summit - sig-platform-dev #### Questions: - Generating docs for CRDs is difficult -- is there a solution for this? - download [Kube-builder](https://github.com/kubernetes-sigs/kubebuilder) and it will do this for you! - client-go: writing a client that could automatically retrieve from a cache or client -- has evolved since it was started. Why are there 2 separate interfaces? - issues: client-set interface allows you to pass special options that allow you to do interesting stuff on the API server which isn't necessarily possible in the lister. - started as same function call and then diverged - lister gives you slice of pointers - clientset gives you a slice of not pointers - a lot of people would take return from clientset and then convert it to a slice of pointers so the listers helped avoid having to do deep copies everytime. TLDR: interfaces are not identical - controllers in languages other than go - checkout [metacontroller](https://github.com/GoogleCloudPlatform/metacontroller)