# Client-go and API extensions
@munnerz
## Goals:
### What is painful when building a controller
#### Questions around best practices
##### After this week:
- Someone new
- What is particularly hard to grasp
- Someone experienced
- What important bits of info are critical?
##### Pain points when building controller
- Boilerplate
- work queues
- HasSynced functions
- Re-queing
- Lack of deep documentation
- some documentation but focused on k/k core
- Securing webhooks & APIServices
- Validation schemas
- Webhooks/secuity is particularly difficult right now
- openshift solution was mentioned
- Scaling API is pretty low level
- Registering CRDs, building and consuming custom controllers
- Do I deploy the CRD as a deployment or have controller register the CRD?
- building/useability is unclear
- declarative API needs to translate to transactional APIs sometimes but there is no best practice/library. Ex. Ingress, specify ingress object
- kubernetes resource model has API types separated
- interaction with third party systems is possible but complicated and needs to be built
- if third party system can't react to change there is no obvious solution. webhooks? not possible for asynchonous
- controllers consuming kubernetes resources that interact with external systems are tricky
##### Best practices - questions
- How many times to update status per sync
- Conditions vs fields
- used to check a pods' state.
- idea behind conditions: you can add conditions that are ok for new clients to pay attention to and old clients to ignore
- Validating webhooks vs OpenAPI schema
- How are people testing extensions?
- anyone resuing upstream dind cluster?
- how else are people testing?
- @pwittrock: kubebuilder: package control plane binaries, generate tests for resources and controllers
- integration tests -- pull in packages that will build the clusters
- kube adm to build single node cluster and running tests against that
- travis ci
- spin up new vms
- Lee from Kubeadm - runs kubeadm on docker and docker node, does multinode and hopes to expand to multicluster: look up if you want to run it against dnd node
- #testing-commons channel on stack. 27 page document on this--link will be put in slides
- Deploying and managing Validating/Mutating webhooks?
- how complex should they be?
- When to use subresources
- Are people switching to api agg to use this today?
- Why not use subresources today with scale?
- multiple replicas fields
- doesn't fit polymorphic structure that exists
- Phil Wittrock: kubectl side, scale
##### Moving forward
- How can we best disseminate this knowledge
- most sigs maintain controllers
- wiki? developer docs WG?
- existing docs focus on in-tree development
- @pwittrock: [Working on a book](book.kubebuilder.io) for Kubebuilder and would appreciate feedback
- Best place to have real-time conversation about these topics is: #sig-apimachinery
- API extension authors meetings/summit
- sig-platform-dev
#### Questions:
- Generating docs for CRDs is difficult -- is there a solution for this?
- download [Kube-builder](https://github.com/kubernetes-sigs/kubebuilder) and it will do this for you!
- client-go: writing a client that could automatically retrieve from a cache or client -- has evolved since it was started. Why are there 2 separate interfaces?
- issues: client-set interface allows you to pass special options that allow you to do interesting stuff on the API server which isn't necessarily possible in the lister.
- started as same function call and then diverged
- lister gives you slice of pointers
- clientset gives you a slice of not pointers
- a lot of people would take return from clientset and then convert it to a slice of pointers so the listers helped avoid having to do deep copies everytime. TLDR: interfaces are not identical
- controllers in languages other than go
- checkout [metacontroller](https://github.com/GoogleCloudPlatform/metacontroller)