Kubernetes Design Patterns part-2

# part-2# K8s Design Patterns ![k8s design patterns](https://i.imgur.com/afSYWTl.png) --- ### Batch Job - Behaviour Pattern - Suited for managing isolated atomic units of work. - Based on Job abstraction - Runs short-lived Pods reliably until completion --- ### 3 Different way of creating Pod 1. Bare Pod 2. Replica Set 3. DeamonSet --- ### Problem - Pods is long-running process that are not meant to stop after some time - There are some cases where we need this. --- ### Solution - Kubernetes Job --- ### Job - Similar to ReplicaSet as it creates one or more Pods and ensure they run successfully - However, the different is that, once the expected number of Pods terminate succesfully, the Job is considered complete --- ### Job Example ```yaml= apiVersion: batch/v1 kind: Job metadata: name: test-job spec: completions: 5 parallelism: 2 template: metadata: name: test-job spec: restartPolicy: OnFailure containers: - image: busybox:1.28.4 name: test-job command: ["sh", "-c", "sleep $((1 + $RANDOM % 10))"] ``` --- ### Benifits of k8s Jobs - When Job completed, its not deleted but kept for tracking purpose - Job may need to perform multiple time using `spec.completions` field, just specify how many successfully completed job you want. - Run Pods in Parallel --- ### Use cases - Some tools uses jobs to run install, setup, or test commands on clusters - Take backup or snapshot - ML model train job - Fine Parallel Processing Using a Work Queue - CI/CD pipelines --- ## Periodic Job Pattern - The Periodic Job pattern extends the Batch Job pattern by adding a time dimension and allowing the execution of a unit of work to be triggered by a temporal event. --- ### Example ```yaml apiVersion: batch/v1beta1 kind: CronJob metadata: name: hello spec: schedule: "*/1 * * * *" jobTemplate: spec: template: spec: containers: - name: hello image: busybox command: ["sh", "-c", "date; echo hello k8s"] restartPolicy: OnFailure ``` --- ### Daemon Service - The Daemon Service pattern allows placing and running prioritized, infrastructure- focused Pods on targeted nodes --- ### Example ```yaml= apiVersion: apps/v1 kind: DaemonSet metadata: name: fake-db spec: selector: matchLabels: app: fake-db template: metadata: labels: app: fake-db spec: nodeSelector: feature: db containers: - image: busybox:1.24.8 name: fake-db command: ["sh", "-c", "sleep 1000"] volumeMounts: - mountPath: /host_dev name: devices volumes: - name: devices hostPath: path: /dev ``` --- ### Create Daemonset ```bash= # Create kubectl create -f ds-file.yml # List kubectl get ds ``` --- ### Label Nodes ```bash= # Add label kubectl label nodes <nodename> feature=db # Show labels kubectl get nodes --show-labels # Remove label kubectl label nodes minikube feature- ``` --- ### Daemonset - By default, places one Pod to every node. Control using `nodeSelector` field. - Its not depend on Scheduler because nodeName is specified - Pods created by DS can run before scheduler started - DS treated high priority - Pods in the DS can specify a hostPort and become reachable via the node IP addresses and the specified port - e.g `kube-proxy` and `kube-dns` are ds --- ### StaticPod - Alternative to DS because kubelet reads local files and deploys it. --- ### Singleton Pattern --- ### Scenario 1# - You have 3 running pods in your cluster but only one Pod write only to Database/Filesystem at a time. - How you will solve this problem?  --- ### Scenario 2# - Poll data from external API but only one connection at a time is allowed  --- ### Things to note - Use Application locking mechanism - e.g. Zookeeper, etcd, consul, apache camel - You cannot rely on the out-of- application locking mechanisms of ReplicaSets, because its not design for this. think about network partition problem. - Leader election with sidecar to do active-passive HA in case of >1 replicas - Pod Disruption Budget Pattern     --- ### Do not Disrupt My Cluster! The Pod Disruption Budget Pattern The Singleton pattern describes a scenario where you need only one instance of your application to be running at any given time. But sometimes, you may also want to ensure that a specific number (or percentage) of your Pods are not unavailable. Despite the existence of controllers like ReplicaSets, Deployments, etc. which ensure that all your Pods are healthy and running, in some scenarios, you are intentionally bringing down a portion of your Pods. The most well-known example of such a scenario is when you are draining a node. Draining refers to gradually removing Pods from a running node till all of them are down. Then, the node can be brought down for maintenance. Beware, though, that Pod Disruption Budget policy only protects against voluntary Pod eviction. If Pods are evicted due to a node becoming unhealthy, PDB is not honored. ### How Does Pod Disruption Work With Other Controllers? Assume that you have a Deployment that spawns five Pod replicas. Now, you need to upgrade one of the cluster nodes. You know that your application can tolerate running with four Pods but no less than that. If one of the four Pods fails, the whole application breaks. This behavior is not uncommon in clustered applications that need to maintain a specific quorum of nodes. So, because of a kernel bug in one of the nodes, you installed a new kernel and you need to restart the node for the new kernel to be used. Restarting the node requires transferring any Pod running on the selected node to other nodes. Without using a Pod Disruption Budget (PDB), Kubernetes just kills all the Pods on the node. If those Pods are managed by a controller like a Deployment or a ReplicaSet, another Pod is automatically spawned and deployed to another healthy node. However, this breaks your application because the quorum is violated. With PDB, the eviction does not happen unless an additional Pod is running so that the minimum number of running Pods is maintained. The following is an example of a Pod Disruption Policy that ensures that at least four Pods of the ones labeled app=mycluster are available at any given time: ```yaml apiVersion: policy/v1beta1 kind: PodDisruptionBudget metadata: name: mycluster-pdb spec: minAvailable: 4 selector: matchLabels: app: mycluster ``` As you can see, the definition starts with the usual data, `apiVersion`, `kind`, and `metadata.name`. The spec part has two important parameters: `minAvailable` (line 6): the required number of Pods that should be kept running at all times. This number only applied to the Pods matched through the Pod selector (line 7). Notice that you can use a percentage instead of a number. For example, 70%. Additionally, you can use `maxUnavailable` instead. The `maxUnavailable` specifies the highest number of Pods that can be missing. Notice that both parameters are mutually exclusive: you can only use one of them but not both. selector: works by searching for all Pods that satisfy a label. It is those Pods on which the PDB policy is applied. ### TL;DR - In many cases, you may need only one instance of your application to be running at a given time. For those application requirements, you should follow the Singleton deployment pattern. - If your application can tolerate brief violations of the Singleton pattern, you can use a ReplicaSet with the number of replicas set to 1. The drawback of this approach is that ReplicaSets tries not to break the availability of the application, even if that means running more Pods than the requested number for a short time period. Possible causes of running another concurrent Pod is when the original Pod runs into issues making it temporarily unavailable, then comes back. - If your application needs to follow a more strict Singleton policy, then you should use a StatefulSet. A StatefulSet does not attempt to spawn a new Pod unless the old one has completely shut down. However, StatefulSets have their own limitations. - Both of the above approaches are not suitable for scaling. You only run one instance of your application on a single Pod and that’s it. So, if your application does other activities that do not require being a Singleton, but only part of it must run solo, you can use an application-managed locking mechanism. In this procedure, the application handles the election of a leader instance. Only the leader instance is allowed to perform the Singleton activity (for example, updating a DB record). When the instance fails, the rest of the instances elect a new leader.