Published
owned this note
owned this note
Linked with GitHub
# K8s CRD/Controllers (1/2)
---
### k8s Master Components
- kube-apiserver
- etcd
- kube-scheduler
- kube-controller-manager
* Node/Replication/Endpoints(Services/Pods)/Service Account & Token
- cloud-controller-manager
---
### What is cloud-providers?
https://kubernetes.io/docs/concepts/architecture/cloud-controller/
- cloud controller manager -> kops
* Node controller
* Route controller
* Service controller
* Volume controller (> CSI)
- for os-in-os cases
---
### OpenStack Cloud Provider
- OpenStack Cloud Controller Manager
- Octavia Ingress Controller
- Cinder CSI Plugin
- Keystone Webhook Authentication - Authorization
- Client Keystone
- Cinder Standalone Provisioner
- Manila CSI Plugin
- Manila Provisioner
- Barbican KMS Plugin
---
### [AWS operator](https://aws.amazon.com/blogs/opensource/aws-service-operator-kubernetes-available/)
- DynamoDB Tables
- S3 Buckets
- Amazon Elastic Container Registry (Amazon ECR)
- Repositories
- SNS Topics
- SQS Queues
- SNS Subscriptions
---
## K8s is API-centric
| Declarative | State | Level based | Transparent |
| ----------- | ------- | ----------- | ----------- |
| What vs How | Desired | Doesn't rely on individual change | One control plane |
| build from config | Observed | Facilities composability| Less prone to error |
| Record changes | | | |
---
### k8s to build a pod/cake?
* Built-in
* node / pod / deployment / service
```
curl localhost:8080/api/v1/namespaces/kube-system/services/ingress/
```
* CRD
* [cake](https://kccncosschn19eng.sched.com/event/NrnG/to-crd-or-not-to-crd-that-is-the-question-ed-king-sam-gunaratne-pivotal)
* room/light
* node/vm
* mysql/Spark/Cassandra
```
curl localhost:8080/apis/apiextensions.k8s.io/v1beta1
```
---
### Stateless is easy, stateful is hard
- k8s can handle stateless
- manage other stateful applications need extra knowledge
- CRD - StatefulSets, Services, ConfigMaps
---
## kubevirt
---
## kubevirt components
- `CRD`
- are added to the Kubernetes API
- `virt-controller`
- cluster wide logic associated with these new types
- `virt-handler`
- node specific logic associated with new types
---
### kubevirt yaml
```
apiVersion: kubevirt.io/v1alpha3
kind: KubeVirt
metadata:
name: kubevirt
namespace: kubevirt
spec:
imagePullPolicy: Always
```
---
### rook - ceph crd
---
### Timeline
[Kubernetes API 与 Operator：不为人知的开发者战争](https://www.lijiaocn.com/%E9%A1%B9%E7%9B%AE/2019/01/08/kubernetes-api-and-operator-history.html)
- v2017/01 v1.6 ThirdPartyResource
- 2017/04 v1.7 CustomResourceDefinition
- 2017/10 v1.9 CustomResourceValidation
- 2018/04 v1.11
- crd subresources /status, /scale
- multiple versions
- redhat http://operatorhub.io/
- knative / https://github.com/keleustes/oslc-operator
---
### crd sdk - pattern/template
- [kubebuilder](https://itnext.io/under-the-hood-of-kubebuilder-framework-ff6b38c10796)
- controller-runtime/ controller-tools: controller-gen/...
- code-generator > create kube api
- [operator-sdk](https://itnext.io/under-the-hood-of-the-operator-sdk-eebc8fdeebbf) [code-generator](https://github.com/kubernetes/code-generator)
- scale, upgrade, or backup
- [metacontroller](https://metacontroller.app/) Lightweight Kubernetes controllers as a service
---
#### kubebuilder
---
#### operator-sdk
---
### operator-sdk devel flow
---
### operator framework
- [到底 Operator Framework 用起來會長得如何?](https://blog.pichuang.com.tw/20190407-how-look-like-operator-framework-in-production/)
---
### Operator Lifycycle Management
---
### Informer list-watch
[Kubernetes Informer 详解](https://www.kubernetes.org.cn/2693.html) / [理解 K8S 的设计精髓之 list-watch](http://wsfdl.com/kubernetes/2019/01/10/list_watch_in_k8s.html) / [理解 K8S 的设计精髓之 List-Watch机制和Informer模块](https://zhuanlan.zhihu.com/p/59660536)
---
---
---
### kubebuilder
```
$ go mod init example.easystack.cn # go v1.11
$ kubebuilder init --domain my.domain
```
---
#### kubebuilder init
```
go mod tidy
Running make...
make
which: no controller-gen in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/shawn/.local/bin:/home/shawn/bin)
go get sigs.k8s.io/controller-tools/cmd/controller-gen@v0.2.0
go: finding sigs.k8s.io/controller-tools/cmd/controller-gen v0.2.0
go: finding sigs.k8s.io/controller-tools/cmd v0.2.0
/home/shawn/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
go fmt ./...
go vet ./...
go build -o bin/manager main.go
Next: Define a resource with:
$ kubebuilder create api
```
---
```
config/
certmanager/ default/ manager/ rbac/ webhook/
Dockerfile
hack
main.go
Makefile
PROJECT
```
---
main.go
```
func main () {
...
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
Scheme: scheme,
MetricsBindAddress: metricsAddr,
LeaderElection: enableLeaderElection,
})
...
// +kubebuilder:scaffold:builder
... // mgr.Start(ctrl.SetupSignalHandler())
}
```
---
#### kubebuilder create api
```
$ kubebuilder create api --group webapp --version v1 --kind Guestbook
Create Resource [y/n]
y
Create Controller [y/n]
y
Writing scaffold for you to edit...
api/v1/guestbook_types.go
controllers/guestbook_controller.go
Running make...
```
* api/v1/guestbook_types.go
* controller/guestbook_controller.go
---
#### kubebuilder apply crd
```
kubectl apply -f config/samples/
```
```
make docker-build docker-push IMG=<some-registry>/controller
make deploy IMG=<some-registry>/controller
```
