# Kubernetes Up and Running, 2nd Edition. GC notes ### What are the main takeaways after finishing the book? 1. Annotations & Labels Labels are used to identify related resources, often identified with a selector query. "Labels are a powerful and ubiquitous glue that holds a Kubernetes application together". Annotations are used for non-identifying information. They keep track of metadata, examples are: most recent update, update strategy, enable extra features, human helpful information, etc. 2. NameSpaces and quotas (or limit ranges) to prevent resources from stepping on each other's toes NameSpaces are a great way to organize your application on a single kubernetes cluster. Different namespaces are separate, but can also talk to one another. Namespacing can help with: Isolation, Organization, Permissions, and Performance. Resource quotas, also known as limits allow you to set maximums to the CPU and memory that a pod can use. 3. Authentication and RBAC (Role-Based Access Control) Kubernetes authenticates every request, this can be done with a pluggable 3rd party authentication provider. Kubernetes gets the user and group of the requester, and uses that info to determine if the requester is authorized to make the request. Role -> Namespaced; ClusterRole -> cluster wide. 4. Stateful applications Stateful apps are applications that stores data from a user's session to be used in a later session. Most applications that an everyday person is used to are stateful. Data is stored for long-term retrieval in a persistent disk storage that is accessible to the server. This is tricky because by default containers are stateless. When you are designing a stateful app, it is important to integrate it correctly, so that data is still accessible if a container restarts. 5. No built-in database, but can set up a connection with an existing or new db. Stateful sets are one way this can be achieved. "Without persistent storage, your application architectures are limited to specific styles, such as twelve-factor apps that aren’t suitable for many enterprise workloads.” We were able to replicate MongoDB with stateful sets. stateful sets are like replica sets, but with indexed names for pods, and they are created in order. These resources, along with a headless (no virtual IP) service to manage DNS. 6. Services and Service Discovery A Service in Kubernetes is an abstraction which defines a set of Pods and a policy by which to access them. This set of pods defined by a services is a loose coupling between dependent pods. The set of Pods targeted by a Service is usually determined by a LabelSelector. Service Discovery is all about finding which processes are listening at which addresses for which services. The service object creates a named label selector. 7. Declarative configuration proof of changes and can be version controlled along the way. You can use your configurations to reset a cluster or hand off directions to someone else for recreation. kubernetes builds on itself a lot, in order to understand deployments, you should know about replicasets, and to understand replicaSets, you need to know about pods. #### Ch 1 Introduction to k8s "Everything in Kubernetes is a declarative configuration object that represents the desired state of the system. It is the job of Kubernetes to ensure that the actual state of the world matches this desired state.[...] declarative configuration is an alternative to imperative configuration, where the state of the world is defined by the execution of a series of instructions rather than a declaration of the desired state of the world. While imperative commands define actions, declarative configurations define state." (p. 4) "Decoupling components via load balancers makes it easy to scale the programs that make up your service, because increasing the size (and therefore the capacity) of the program can be done without adjusting or reconfiguring any of the other layers of your service." (p. 6) "an important aspect of this decoupling is that the container orchestration API becomes a crisp contract that separates the responsibilities of the application operator from the cluster orchestration operator. We call this the “not my monkey, not my circus” line. The application developer relies on the service-level agreement (SLA) delivered by the container orchestration API, without worrying about the details of how this SLA is achieved. Likewise, the container orchestration API reliability engineer focuses on delivering the orchestration API’s SLA without worrying about the applications that are running on top of it." (p. 8) #### ch 2 Creating and Running Containers "These programs accept input, manipulate data, and then return the results. Before we can even consider building a distributed system, we must first consider how to build the application container images that contain these programs and make up the pieces of our distributed system." (p. 13) #### ch 3 Deploying a Cluster "The Kubernetes proxy is responsible for routing network traffic to load-balanced services in the Kubernetes cluster. To do its job, the proxy must be present on every node in the cluster. Kubernetes has an API object named DaemonSet, which you will learn about later in the book, that is used in many clusters to accomplish this." (p. 34) #### ch 4 Kubectl commands "Let’s assume that you have a simple object stored in obj.yaml. You can use kubectl to create this object in Kubernetes by running: $ kubectl apply -f obj.yaml Notice that you don’t need to specify the resource type of the object; it’s obtained from the object file itself. Similarly, after you make changes to the object, you can use the apply command again to update the object: $ kubectl apply -f obj.yaml" (p. 39) #### ch 5 Pods "By separating the two applications into two separate containers, we can ensure reliable web server operation." (p. 46) " If you group the WordPress and MySQL containers together in a single Pod, you are forced to use the same scaling strategy for both containers, which doesn’t fit well." (p. 47) "Liveness health checks run application-specific logic (e.g., loading a web page) to verify that the application is not just still running, but is functioning properly. Since these liveness health checks are application-specific, you have to define them in your Pod manifest." (p. 54) "Many applications are stateful, and as such we must preserve any data and ensure access to the underlying storage volume regardless of what machine the application runs on. As we saw earlier, this can be achieved using a persistent volume backed by network-attached storage. We also want to ensure that a healthy instance of the application is running at all times, which means we want to make sure the container running kuard is ready before we expose it to clients. Through a combination of persistent volumes, readiness and liveness probes, and resource restrictions, Kubernetes provides everything needed to run stateful applications reliably. (p. 61-62) #### ch 6 Labels and Annotations "For example, ReplicaSets, which create and maintain multiple replicas of a Pod, find the Pods that they are managing via a selector. Likewise, a service load balancer finds the Pods it should bring traffic to via a selector query. When a Pod is created, it can use a node selector to identify a particular set of nodes that it can be scheduled onto. When people want to restrict network traffic in their cluster, they use NetworkPolicy in conjunction with specific labels to identify Pods that should or should not be allowed to communicate with each other. Labels are a powerful and ubiquitous glue that holds a Kubernetes application together. Though your application will likely start out with a simple set of labels and queries, you should expect it to grow in size and sophistication with time" (p. 61) "Annotations are used to: • Keep track of a “reason” for the latest update to an object. • Communicate a specialized scheduling policy to a specialized scheduler. • Extend data about the last tool to update the resource and how it was updated (used for detecting changes by other tools and doing a smart merge). • Attach build, release, or image information that isn’t appropriate for labels (may include a Git hash, timestamp, PR number, etc.). • Enable the Deployment object (Chapter 10) to keep track of ReplicaSets that it is managing for rollouts. • Provide extra data to enhance the visual quality or usability of a UI. For example, objects could include a link to an icon (or a base64-encoded version of an icon). • Prototype alpha functionality in Kubernetes (instead of creating a first-class API field, the parameters for that functionality are encoded in an annotation)." (p. 72) #### ch 7 Service Discovery "Service discovery tools help solve the problem of finding which processes are listening at which addresses for which services. A good service-discovery system will enable users to resolve this information quickly and reliably. A good system is also low-latency; clients are updated soon after the information associated with a service changes. Finally, a good service-discovery system can store a richer definition of what that service is. For example, perhaps there are multiple ports associated with the service." (p. 75) "The DNS service is, itself, managed by Kubernetes and is a great example of Kubernetes building on Kubernetes. The Kubernetes DNS service provides DNS names for cluster IPs." (p. 77) #### ch 8 HTTP Load balancing with Ingress "Users can create and modify Ingress objects just like every other object. But, by default, there is no code running to actually act on those objects. It is up to the users (or the distribution they are using) to install and manage an outside controller. In this way, the controller is pluggable." (p. 90) #### ch 9 ReplicaSets "With a ReplicaSet, it is the desired number of replicas and the definition of the Pod to replicate. For example, the desired state is that there are three replicas of a Pod running the kuard server." (p. 104) "Sets use label queries to identify the set of Pods they should be managing. They then use the exact same Pod API that you used directly in Chapter 5 to create the Pods that they are managing. This notion of “coming in the front door” is another central design concept in Kubernetes. In a similar decoupling, ReplicaSets that create multiple Pods and the services that load-balance to those Pods are also totally separate, decoupled API objects. In addition to supporting modularity, the decoupling of Pods and ReplicaSets enables several important behaviors" (p. 105) #### ch 10 Deployments "Let’s explore how deployments actually work. Just as we learned that ReplicaSets manage Pods, deployments manage ReplicaSets. As with all relationships in Kubernetes, this relationship is defined by labels and a label selector. You can see the label selector by looking at the Deployment object" (p. 114) #### ch 11 DaemonSets #### ch 12 Jobs "After the job has completed, the Job object and related Pod are still around. This is so that you can inspect the log output. Note that this job won’t show up in kubectl get jobs unless you pass the -a flag. Without this flag, kubectl hides completed jobs" (p. 141) #### ch 13 ConfigMaps and Secrets "By default, Kubernetes secrets are stored in plain text in the etcd storage for the cluster. Depending on your requirements, this may not be sufficient security for you. In particular, anyone who has cluster administration rights in your cluster will be able to read all of the secrets in the cluster. In recent versions of Kubernetes, support has been added for encrypting the secrets with a user-supplied key, generally integrated into a cloud key store. Additionally, most cloud key stores have integration with Kubernetes flexible volumes, enabling you to skip Kubernetes secrets entirely and rely exclusively on the cloud provider’s key store." (p. 158) "Secrets and ConfigMaps are managed through the Kubernetes API. The usual create, delete, get, and describe commands work for manipulating these objects." (p. 162) #### ch 14 Role-Based Access Control for Kubernetes "Every request to Kubernetes is first authenticated. Authentication provides the identity of the caller issuing the request. It could be as simple as saying that the request is unauthenticated, or it could integrate deeply with a pluggable authentication provider (e.g., Azure Active Directory) to establish an identity within that third-party system. Interestingly enough, Kubernetes does not have a built-in identity store, focusing instead on integrating other identity sources within itself." (p. 168) "You cannot use namespaced roles for non-namespaced resources (e.g., CustomResourceDefinitions), and binding a Role Binding to a role only provides authorization within the Kubernetes namespace that contains both the Role and the RoleDefinition." (p. 169) "Of course, sometimes you want to create a role that applies to the entire cluster, or you want to limit access to cluster-level resources. To achieve this, you use the ClusterRole and ClusterRoleBinding resources. They are largely identical to their namespaced peers, but with larger scope" (p. 170) #### ch 15 Integrating Storage Solution and Kubernetes "Integrating this data with containers and container orchestration solutions is often the most complicated aspect of building a distributed system. This complexity largely stems from the fact that the move to containerized architectures is also a move toward decoupled, immutable, and declarative application development. These patterns are relatively easy to apply to stateless web applications, but even “cloud-native” storage solutions like Cassandra or MongoDB involve some sort of manual or imperative steps to set up a reliable, replicated solution" (p. 177) "When you deploy a Pod into the test namespace and it looks up the service named my-database, it will receive a pointer to my-database.test.svc.cluster.internal, which in turn points to the test database. In contrast, when a Pod deployed in the prod namespace looks up the same name (my-database) it will receive a pointer to my-database.prod.svc.cluster.internal, which is the production database. Thus, the same service name, in two different namespaces, resolves to two different services" (p. 179) "Once the StatefulSet is created, we also need to create a “headless” service to manage the DNS entries for the StatefulSet. In Kubernetes a service is called “headless” if it doesn’t have a cluster virtual IP address. Since with StatefulSets each Pod has a unique identity, it doesn’t really make sense to have a load-balancing IP address for the replicated service. " (p. 188) #### ch 16 Extending Kubernetes "extensions to the API server via adding new resource types or admission controllers to API requests... Admission controllers are called prior to the API object being written into the backing storage. Admission controllers can reject or modify API requests. There are several admission controllers that are built into the Kubernetes API server; for example, the limit range admission controller that sets default limits for Pods without them. Many other systems use custom admission controllers to auto-inject sidecar containers into all Pods created on the system to enable “auto-magic” experiences." (p. 196) "One thing that you’ll note is that we never defined the schema for the custom resource in the CustomResourceDefinition. It actually is possible to provide an OpenAPI specification for a custom resource, but this complexity is generally not worth it for simple resource types." (p. 198) #### ch 17 Deploying Real-World Applications "a reliable Redis installation actually is two programs working together. The first is redis-server, which implements the key/value store, and the other is redis-sentinel, which implements health checking and failover for a replicated Redis cluster." (p. 214) #### ch 18 Organizing Your Application "reliability and agility are the general goals of developing a cloud-native application in Kubernetes, but moving to the next level of detail, how does this actually relate to how you design the maintenance and deployment of your application? The following sections describe the various principles that we can use as a guide to design a structure that best suits these goals. The principles are: • Filesystems as the source of truth • Code review to ensure the quality of changes • Feature flags for staged roll forward and roll back" (p. 221)