Lab Session One

# Lab Session One > These documents are my working notes for each of our weekly sessions. Think of them as a Cliff's Notes version of Kubernetes the Hard Way with some of my own commentary. If we find anything substantive in our sessions, we'll be sure to open and link Pull Requests against the KTHW source of truth on Github. > > -jduncan ## Persistent Notes and links * Kubernetes The Hard Way - https://github.com/kelseyhightower/kubernetes-the-hard-way * GDoc (internal to Google) - [go/k8s-appreciation-month](go/k8s-appreciation-month) * This project in Github - https://github.com/jduncan-rva/k8s-appreciation-month * Session One - https://hackmd.io/6NFDYrWdQC677I-arkQWmg * Session Two - https://hackmd.io/RIEmIlXpRouNlVKhBQ3Thw * Session Three - https://hackmd.io/xs65EKFhRi-PnEDwBdzSYQ * Session Four - https://hackmd.io/S62A412aQb2Q5pBsVsDqgg ## Weclome to Kubernetes Appreciation Month! In our first session we'll be covering these labs from [Kubernetes The Hard Way](https://github.com/kelseyhightower/kubernetes-the-hard-way). * [Prerequisites](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/01-prerequisites.md) * [Installing the Client Tools](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/02-client-tools.md) * [Provisioning Compute Resources](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/03-compute-resources.md) * [Provisioning the CA and Generating TLS Certificates](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md) ## Prerequisites [*KTHW Link*](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/01-prerequisites.md) for this lab content. ### Configuring Google Cloud SDK 1. Test the version of Google Cloud SDK on your system. You'll need to update if you're running anything below 301.0.0. `gcloud version` 2. Run `gcloud init` to make sure you're authenticated with the proper account. If you're new to the Google SDK, this will help you set up everything. If you're experienced with `gcloud`, this step may be optional. `gcloud init` If you're setting up a new account for the first time you'll be given the option to select a default zone and region after you log in. If you do that as part of this command, you can skip doing it in steps 3-5. 3. Once you're account is set up, run `gcloud auth login` to make sure you're logged in as the proper user. `gcloud auth login` 4. Check the currently available regions in GCP with the following command. Pick something geographically close to where you're located. `gcloud compute zones list` I'll be using the `us-east4-a` zone in the `us-east4` region. Note that if this is a new project, you may be prompted to enable the compute API. This is normal. 5. Once you've picked a region and zone, tell `gcloud` to use them with these commands: ``` gcloud config set compute/region $YOUR_REGION gcloud config set compute/zone $YOUR_ZONE ``` With your configuration set, you're ready to decide how you want to handle multiple simultaneous SSH sessions to configure your cluster. ### Session Handling Throughout the labs for K8s-Apprecation-Month you'll be accessing multiple hosts simultaneously to configure your Kubernetes cluster. A multiplexing tool like [`tmux`](https://github.com/tmux/tmux/wiki). There are many ways to install `tmux` on to Linux and MacOS systems. There's probably even a version that works with WSL on Windows 10, but that's far outside of our scope for this lab of labs. If you don't want to use `tmux`, most modern terminal emulators can do multiple panes or tabs. Use whatever you're comfortable with. With `gcloud` configured and your session management preferences ready to go, Lab 1 is complete we're ready to get some additional client tooling ready. ## Client Tools [*KTHW Link*](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/02-client-tools.md) for this lab content. Kubernetes uses [TLS encryption](https://en.wikipedia.org/wiki/Transport_Layer_Security) extensively. To create the needed encryption keys and certificates we'll use Clouflare's free tools `cfssl` and `cfssljson`. Additionally we'll install `kubectl`, the primary command line tool to interface with kubernetes in this lab. ### Installing PKI infrastructure *This is taken almost verbatim from the KTHW lab link* The `cfssl` and `cfssljson` command line utilities will be used to provision a [PKI Infrastructure](https://en.wikipedia.org/wiki/Public_key_infrastructure) and generate TLS certificates. Download and install `cfssl` and `cfssljson`: #### OS X 1. Download the OSX binaries ``` curl -o cfssl https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/darwin/cfssl curl -o cfssljson https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/darwin/cfssljson ``` 2. Make the downloaded files executable ``` chmod +x cfssl cfssljson ``` 3. Move the files to a directory in $PATH ``` sudo mv cfssl cfssljson /usr/local/bin/ ``` Some OS X users may experience problems using the pre-built binaries in which case [Homebrew](https://brew.sh) might be a better option: ``` brew install cfssl ``` #### Linux 1. Download the Linux binaries ``` wget -q --show-progress --https-only --timestamping \ https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/linux/cfssl \ https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/1.4.1/linux/cfssljson ``` 2. Make the Linux binaries executable ``` chmod +x cfssl cfssljson ``` 3. Move the files to a directory in $PATH. If you're using Cloud Shell, don't do this step and move to the next section. ``` sudo mv cfssl cfssljson /usr/local/bin/ ``` #### GCP Cloud Shell If you're using a GCP Cloud Shell instance for this lab, a small change to the Linux steps above will make your life easier. Steps 1 and 2 remain the same. But instead of step 3, run the following: 1. Create a `bin` directory in your `$HOME` directory ``` mkdir $HOME/bin ``` 2. Add your new directory to your `$PATH` variable. ``` export PATH=$PATH:$HOME/bin ``` to have this persist across Cloud Shell restarts, you'll need to add this command to your `~/.bashrc` or other shell `.rc` file. That is a simple task, but out of scope for this workshop. 3. Move `cfssl` and `cfssljson` to `$HOME/bin` ``` mv cfssl cfssljson $HOME/bin/ ``` You're now ready to move on to the next section and verify the functionality of your new tools. #### Verifying cfssl and cfssljson To verify `cfssl` and `cfssljson` version 1.4.1 or higher is installed. command: ``` cfssl version ``` output: ``` Version: 1.4.1 Runtime: go1.12.12 ``` command: ``` cfssljson --version ``` output: ``` Version: 1.4.1 Runtime: go1.12.12 ``` ### Installing kubectl *This is taken almost verbatim from the KTHW lab link* The `kubectl` command line utility is used to interact with the Kubernetes API Server. If you're using a Cloud Shell instance, `kubectl` is already installed. For other systems, download and install `kubectl` from the official release binaries: #### OS X 1. Download the kubectl OSX binary ``` curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.18.6/bin/darwin/amd64/kubectl ``` 2. Make `kubectl` executable ``` chmod +x kubectl ``` 3. Move `kubectl` to a directory in your $PATH variable ``` sudo mv kubectl /usr/local/bin/ ``` #### Linux 1. Download the Linux binary ``` wget https://storage.googleapis.com/kubernetes-release/release/v1.18.6/bin/linux/amd64/kubectl ``` 2. Make the file excutable ``` chmod +x kubectl ``` 3. Move `kubectl` into a directory in $PATH ``` sudo mv kubectl /usr/local/bin/ ``` #### Verification Verify `kubectl` version 1.18.6 or higher is installed: ``` kubectl version --client ``` sample output: ``` Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:58:53Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"} ``` With your PKI tools and `kubectl` installed you have the tools you need to deploy your cluster. The next lab provisions your compute instances in GCP that will become your Kubernetes cluster. ## Provisioning Compute and Networking [*KTHW Lab Link*](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/03-compute-resources.md) The Kubernetes cluster we're designing and deploying will exist in the single zone you configured as your default zone in the [Prerequisites Lab](https://hackmd.io/6NFDYrWdQC677I-arkQWmg?both#Prerequisites). In that zone we'll begin by deploying a VPC for the cluster. ### Cluster Networking Kubernetes assumes simple Layer 3 routability between all nodes in its cluster. Latency between control plane nodes is best kept under 10ms. Latency between application nodes or between application nodes and control plane nodes can be higher; some people have had success with latency as high as 150ms. But simple is almost always best. So we'll place all of our nodes in a single VPC in a single zone. > There are lots of ways to implement the [networking model](https://kubernetes.io/docs/concepts/cluster-administration/networking) in Kubernetes. For our infrastructure, we'll leverage the [GCE model implementation](https://kubernetes.io/docs/concepts/cluster-administration/networking/#google-compute-engine-gce). > > When you need complex restrictions between nodes in your cluster, you can leverage [Network Policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/). While really cool, they're out of scope for our efforts. #### VPC Configuration To properly configure the VPC: 1. Provision the VPC ``` gcloud compute networks create kubernetes-the-hard-way --subnet-mode custom ``` output: ``` Created [https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/global/networks/kubernetes-the-hard-way]. NAME SUBNET_MODE BGP_ROUTING_MODE IPV4_RANGE GATEWAY_IPV4 kubernetes-the-hard-way CUSTOM REGIONAL ``` Instances on this network will not be reachable until firewall rules are created. As an example, you can allow all internal traffic between instances as well as SSH, RDP, and ICMP by running: ``` $ gcloud compute firewall-rules create <FIREWALL_NAME> --network kubernetes-the-hard-way --allow tcp,udp,icmp --source-ranges <IP_RANGE> $ gcloud compute firewall-rules create <FIREWALL_NAME> --network kubernetes-the-hard-way --allow tcp:22,tcp:3389,icmp ``` 2. Create a subnet in the VPC ``` gcloud compute networks subnets create kubernetes \ --network kubernetes-the-hard-way \ --range 10.240.0.0/24 ``` output: ``` Created [https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/regions/us-east4/subnetworks/kubernetes]. NAME REGION NETWORK RANGE kubernetes us-east4 kubernetes-the-hard-way 10.240.0.0/24 ``` This subnet can host up to 254 compute hosts. But you won't need quite that many for this lab. 3. Create a firewall rule for the VPC to allow unrestricted internal communications for all protocols. ``` gcloud compute firewall-rules create kubernetes-the-hard-way-allow-internal \ --allow tcp,udp,icmp \ --network kubernetes-the-hard-way \ --source-ranges 10.240.0.0/24,10.200.0.0/16 ``` output: ``` Creating firewall...⠹Created [https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/global/firewalls/kubernetes-the-hard-way-allow-internal]. Creating firewall...done. NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED kubernetes-the-hard-way-allow-internal kubernetes-the-hard-way INGRESS 1000 tcp,udp,icmp False ``` This is not a production-ready configuration. This is not a cluster that you should take into production! 4. Create a VPC firewall rule to allow inbound SSH, ICMP, and HTTPS ``` gcloud compute firewall-rules create kubernetes-the-hard-way-allow-external \ --allow tcp:22,tcp:6443,icmp \ --network kubernetes-the-hard-way \ --source-ranges 0.0.0.0/0 ``` output: ``` Creating firewall...⠹Created [https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/global/firewalls/kubernetes-the-hard-way-allow-external]. Creating firewall...done. NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED kubernetes-the-hard-way-allow-external kubernetes-the-hard-way INGRESS 1000 tcp:22,tcp:6443,icmp False ``` Later, you'll configure a [GCP Load Balancer](https://cloud.google.com/load-balancing/docs/network) to access the Kubernetes API server. 5. Allocate a Static IP for the External Load Balancer. ``` gcloud compute addresses create kubernetes-the-hard-way \ --region $(gcloud config get-value compute/region) ``` output: ``` Created [https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/regions/us-east4/addresses/kubernetes-the-hard-way]. ``` 6. Verify the firewall rules are in place. ``` gcloud compute firewall-rules list --filter="network:kubernetes-the-hard-way" ``` output: ``` NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED kubernetes-the-hard-way-allow-external kubernetes-the-hard-way INGRESS 1000 tcp:22,tcp:6443,icmp False kubernetes-the-hard-way-allow-internal kubernetes-the-hard-way INGRESS 1000 tcp,udp,icmp False ``` 7. Verify the Static IP address has been allocated. ``` gcloud compute addresses list --filter="name=('kubernetes-the-hard-way')" ``` output: ``` NAME ADDRESS/RANGE TYPE PURPOSE NETWORK REGION SUBNET STATUS kubernetes-the-hard-way XX.XXX.XXX.XXX EXTERNAL us-east4 RESERVED ``` With your firewall rules and static IP address properly configured, the networking resources for your cluster are ready. Next, you'll configure your cluster's compute instances. ### Cluster Compute Nodes The nodes in this Kubernetes cluster will run [Ubuntu Server](https://www.ubuntu.com/server) 20.04 because it has solid support for the [containerd container runtime](https://github.com/containerd/containerd). Each node will be configured with a static internal IP address to make the kubernetes bootstrapping process a little simpler. 1. Deploy 3 control plane nodes ``` for i in 0 1 2; do gcloud compute instances create controller-${i} \ --async \ --boot-disk-size 200GB \ --can-ip-forward \ --image-family ubuntu-2004-lts \ --image-project ubuntu-os-cloud \ --machine-type e2-standard-2 \ --private-network-ip 10.240.0.1${i} \ --scopes compute-rw,storage-ro,service-management,service-control,logging-write,monitoring \ --subnet kubernetes \ --tags kubernetes-the-hard-way,controller done ``` output: ``` NOTE: The users will be charged for public IPs when VMs are created. Instance creation in progress for [controller-0]: https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/zones/us-east4-a/operations/operation-1601315596631-5b063574f7cf3-5af5893b-c22194db Use [gcloud compute operations describe URI] command to check the status of the operation(s). NOTE: The users will be charged for public IPs when VMs are created. Instance creation in progress for [controller-1]: https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/zones/us-east4-a/operations/operation-1601315599423-5b063577a19d6-3db6dc50-475fb775 Use [gcloud compute operations describe URI] command to check the status of the operation(s). NOTE: The users will be charged for public IPs when VMs are created. Instance creation in progress for [controller-2]: https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/zones/us-east4-a/operations/operation-1601315601372-5b0635797d712-2b0edb32-8fed374d Use [gcloud compute operations describe URI] command to check the status of the operation(s). ``` 2. Deploy 3 application nodes ``` for i in 0 1 2; do gcloud compute instances create worker-${i} \ --async \ --boot-disk-size 200GB \ --can-ip-forward \ --image-family ubuntu-2004-lts \ --image-project ubuntu-os-cloud \ --machine-type e2-standard-2 \ --metadata pod-cidr=10.200.${i}.0/24 \ --private-network-ip 10.240.0.2${i} \ --scopes compute-rw,storage-ro,service-management,service-control,logging-write,monitoring \ --subnet kubernetes \ --tags kubernetes-the-hard-way,worker done ``` output: ``` NOTE: The users will be charged for public IPs when VMs are created. Instance creation in progress for [worker-0]: https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/zones/us-east4-a/operations/operation-1601315687228-5b0635cb5e63b-ff7afeaa-d8a293f0 Use [gcloud compute operations describe URI] command to check the status of the operation(s). NOTE: The users will be charged for public IPs when VMs are created. Instance creation in progress for [worker-1]: https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/zones/us-east4-a/operations/operation-1601315689247-5b0635cd4b352-11d440dc-e33f84b6 Use [gcloud compute operations describe URI] command to check the status of the operation(s). NOTE: The users will be charged for public IPs when VMs are created. Instance creation in progress for [worker-2]: https://www.googleapis.com/compute/v1/projects/kthw-jduncan-dev/zones/us-east4-a/operations/operation-1601315691256-5b0635cf35be8-277d8bfd-6095f14b Use [gcloud compute operations describe URI] command to check the status of the operation(s). ``` The `metadata` parameter for your application nodes defines the subnet used for the pods on each node. They must be part of the network defined by the `--cluster-cidr` for the `kubernetes-controller-manager` service. For this lab, that value will be `10.200.0.0/16`, and you'll configure it in a later lab. 3. Verify these nodes are all functional: ``` gcloud compute instances list --filter="tags.items=kubernetes-the-hard-way" ``` output: ``` NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS controller-0 us-east4-a e2-standard-2 10.240.0.10 35.245.213.69 RUNNING controller-1 us-east4-a e2-standard-2 10.240.0.11 34.86.61.230 RUNNING controller-2 us-east4-a e2-standard-2 10.240.0.12 35.199.31.109 RUNNING worker-0 us-east4-a e2-standard-2 10.240.0.20 35.245.243.21 RUNNING worker-1 us-east4-a e2-standard-2 10.240.0.21 34.86.224.225 RUNNING worker-2 us-east4-a e2-standard-2 10.240.0.22 34.86.19.61 RUNNING ``` ### Accessing nodes with SSH You'll use SSH to continue configuring your cluster from here. The `gcloud` utility can automatically store your SSH keys for hosts. To test out connectivity, use `gcloud` to connect to `controller-0` via ssh. There's not a need to enter a password for this SSH key. It will only be used for this lab. 1. Connect to controller-0 via ssh ``` gcloud compute ssh controller-0 ``` output: ``` WARNING: The public SSH key file for gcloud does not exist. WARNING: The private SSH key file for gcloud does not exist. WARNING: You do not have an SSH key for gcloud. WARNING: SSH keygen will be executed to generate a key. Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/$USER/.ssh/google_compute_engine. Your public key has been saved in /home/$USER/.ssh/google_compute_engine.pub. The key fingerprint is: SHA256:nz1i8jHmgQuGt+WscqP5SeIaSy5wyIJeL71MuV+QruE $USER@$HOSTNAME The key's randomart image is: +---[RSA 2048]----+ | | | | | | | . | |o. oS | |=... .o .o o | |+.+ =+=.+.X o | |.+ ==O*B.B = . | | .+.=EB++ o | +----[SHA256]-----+ Updating project ssh metadata...-Updated [https://www.googleapis.com/compute/v1/projects/$PROJECT_ID]. Updating project ssh metadata...done. Waiting for SSH key to propagate. ``` Once this process finishes you'll be connected to `controller-0`. ``` Welcome to Ubuntu 20.04 LTS (GNU/Linux 5.4.0-1019-gcp x86_64) ... ``` 2. Close the ssh session to controller-0 ``` exit ``` Your compute and networking resources are ready to go at this point. Next you'll begin to configure the software that will be your kubernetes cluster. You'll by creating a certificate authority and a whole load of TLS certificates. ## Certificate Authority and TLS Certificates [*KTHW Link*](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md) By design, all traffic in and out of Kubernetes control plane services are encrypted with unique TLS certificates. Those services are: * kube-api-server * kube-scheduler * kubelet * kube-proxy To create and manage all these keys and certificates, you'll use the Cloudflare PKI toolkit you installed in the [Prerequisites Lab](https://hackmd.io/6NFDYrWdQC677I-arkQWmg?both#Prerequisites) to bootstrap a certificate authority (CA) and generate the needed certificates. Once created, you'll distribute them to their proper cluster node. ### Creating a Certificate Authority 1. Create ca-config.json ``` cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "8760h" }, "profiles": { "kubernetes": { "usages": ["signing", "key encipherment", "server auth", "client auth"], "expiry": "8760h" } } } } EOF ``` 2. Create ca-csr.json ``` cat > ca-csr.json <<EOF { "CN": "Kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Raleigh", "O": "Kubernetes", "OU": "NC", "ST": "North Carolina" } ] } EOF ``` 3. Create your certificate authority ``` cfssl gencert -initca ca-csr.json | cfssljson -bare ca ``` 4. Verify your certificate was created properly ``` openssl x509 -text -noout -in ca.pem ``` output: ``` Certificate: Data: Version: 3 (0x2) Serial Number: ... ``` #### Generated files * `ca-config.json` * `ca-csr.json` * `ca.csr` * `ca-key.pem` * `ca.pem` This key and certificate files are used to sign the rest of the certificates you create for your cluster. They are your certificate authority. ### Creating the admin client certificate Each kubernetes service needs a client and server certificate. Additionally, the default `admin` user needs a client certificate to be able to authenticate to the cluster's control plane. 1. Create the Certificate Signing Request (CSR) file ``` cat > admin-csr.json <<EOF { "CN": "admin", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Raleigh", "O": "system:masters", "OU": "Kubernetes The Hard Way", "ST": "North Carolina" } ] } EOF ``` 2. Generate the signed key and certificate from the CSR file ``` cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ admin-csr.json | cfssljson -bare admin ``` 3. Verify your admin certificate ``` openssl x509 -text -noout -in admin.pem ``` output: ``` Certificate: Data: Version: 3 (0x2) Serial Number: ... ``` #### Generated files * `admin-key.pem` * `admin.pem` * `admin.csr` * `admin-csr.json` ### Creating kubelet client certificates The `kubelet` service runs on each node in your cluster. Kubelet interfaces with the container runtime on each node and reports information back to the control plane. To authoritze a node to be part of a cluster, `kubelet` uses a credential for an [authentication mechanism](https://kubernetes.io/docs/admin/authorization/node/) that confirms they're part of the `system:nodes` group. Each worker node needs a unique certificate that you'll create in this sexction. This certificate needs to reference both the internal and exteral IP addresses for each node. This will allow for all needed communications. The snippet below does the following for each node: * Creates a CSR file * Captures the internal IP address as a variable * Captures the external IP addresses as a variable * Uses both IPs and the CSR to create a TLS certificate 1. Run the following small script. ``` for instance in worker-0 worker-1 worker-2; do cat > ${instance}-csr.json <<EOF { "CN": "system:node:${instance}", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Raleigh", "O": "system:nodes", "OU": "Kubernetes The Hard Way", "ST": "North Carolina" } ] } EOF EXTERNAL_IP=$(gcloud compute instances describe ${instance} \ --format 'value(networkInterfaces[0].accessConfigs[0].natIP)') INTERNAL_IP=$(gcloud compute instances describe ${instance} \ --format 'value(networkInterfaces[0].networkIP)') cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -hostname=${instance},${EXTERNAL_IP},${INTERNAL_IP} \ -profile=kubernetes \ ${instance}-csr.json | cfssljson -bare ${instance} done ``` 2. Verify one of your kubelet certificates. ``` openssl x509 -text -noout -in worker-0.pem ``` output: ``` Certificate: Data: Version: 3 (0x2) Serial Number: ``` #### Generated files * `worker-0-key.pem` * `worker-0.pem` * `worker-1-key.pem` * `worker-1.pem` * `worker-2-key.pem` * `worker-2.pem` ### Creating the kube-controller-manager client certificate This certificate will be used by clients communicating with the `kube-controller-manager` service. 1. Generate the CSR file ``` cat > kube-controller-manager-csr.json <<EOF { "CN": "system:kube-controller-manager", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Raleigh", "O": "system:kube-controller-manager", "OU": "Kubernetes The Hard Way", "ST": "North Carolina" } ] } EOF ``` 2. Generate the signed certificate and key file ``` cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager ``` 3. Verify the `kube-controller-manager` certificate ``` openssl x509 -text -noout -in kube-controller-manager.pem ``` output: ``` Certificate: Data: Version: 3 (0x2) Serial Number: ... ``` #### Generated files * `kube-controller-manager-key.pem` * `kube-controller-manager.pem` * `kube-controller-manager.csr` * `kube-controller-manager-csr.json` ### Creating the kube-proxy client certificate This certificate is used to communicate with the `kube-proxy` service. 1. Generate the CSR file ``` cat > kube-proxy-csr.json <<EOF { "CN": "system:kube-proxy", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Raleigh", "O": "system:node-proxier", "OU": "Kubernetes The Hard Way", "ST": "North Carolina" } ] } EOF ``` 2. Generate the signed certificate and key file ``` cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ kube-proxy-csr.json | cfssljson -bare kube-proxy ``` 3. Verify the `kube-proxy` certificate. ``` openssl x509 -text -noout -in kube-proxy.pem ``` output: ``` Certificate: Data: Version: 3 (0x2) Serial Number: ... ``` #### Generated files * `kube-proxy-key.pem` * `kube-proxy.pem` * `kube-prox-csr.json` * `kube-proxy.csr` ### Creating the kube-scheduler client certificates This certificate is used to communicate with the `kube-scheduler` control plane service. 1. Generate the CSR file ``` cat > kube-scheduler-csr.json <<EOF { "CN": "system:kube-scheduler", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Raleigh", "O": "system:kube-scheduler", "OU": "Kubernetes The Hard Way", "ST": "North Carolina" } ] } EOF ``` 2. Generate the signed certificate and key file ``` cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ kube-scheduler-csr.json | cfssljson -bare kube-scheduler ``` 3. Verify the `kube-scheduler` certificate ``` openssl x509 -text -noout -in kube-scheduler.pem ``` output: ``` Certificate: Data: Version: 3 (0x2) Serial Number: ... ``` #### Generated files * `kube-scheduler-key.pem` * `kube-scheduler.pem` * `kube-scheduler-csr.json` * `kube-scheduler.csr` ### Creating the kubernetes API server certificate To allow for valid access to your cluster, the kubernetes-the-hard-way static IP address will be included in the list of subject alternative names for the Kubernetes API Server certificate. This will make it a valid TLS certificate. 1. Generate the CSR file ``` cat > kubernetes-csr.json <<EOF { "CN": "kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Raleigh", "O": "Kubernetes", "OU": "Kubernetes The Hard Way", "ST": "North Carolina" } ] } EOF ``` 2. Set the kubernetes-the-hard-way public IP address as a variable ``` KUBERNETES_PUBLIC_ADDRESS=$(gcloud compute addresses describe kubernetes-the-hard-way \ --region $(gcloud config get-value compute/region) \ --format 'value(address)') ``` You can verify this worked by checking the value of the variable. ``` echo $KUBERNETES_PUBLIC_ADDRESS 35.194.79.116 ``` 3. Set all of the valid internal hostnames for kubernetes to a variable. The Kubernetes API server is automatically assigned the kubernetes internal dns name, which will be linked to the first IP address (10.32.0.1) from the address range (10.32.0.0/24) reserved for internal cluster services during the control plane bootstrapping lab. ``` KUBERNETES_HOSTNAMES=kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.svc.cluster.local ``` 4. Generate the signed certificates and key file ``` cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -hostname=10.32.0.1,10.240.0.10,10.240.0.11,10.240.0.12,${KUBERNETES_PUBLIC_ADDRESS},127.0.0.1,${KUBERNETES_HOSTNAMES} \ -profile=kubernetes \ kubernetes-csr.json | cfssljson -bare kubernetes ``` 5. Verify the certificate was created properly. Be sure that your alternative name values are correct, or it could cause problems later. ``` openssl x509 -text -noout -in kubernetes.pem Certificate: Data: Version: 3 (0x2) Serial Number: ... X509v3 Subject Alternative Name: DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster, DNS:kubernetes.svc.cluster.local, IP Address:10.32.0.1, IP Address:10.240.0.10, IP Address:10.240.0.11, IP Address:10.240.0.12, IP Address:35.194.79.116, IP Address:127.0.0.1 ``` #### Generated files * `kubernetes-key.pem` * `kubernetes.pem` * `kubenetes-csr.json` * `kubernetes.csr` ### Creating the service account key pair The `kubernetes-controller-manager` service uses a key pair to generate and sign service account tokens as described in the [managing service accounts](https://kubernetes.io/docs/admin/service-accounts-admin/) documentation. 1. Generate the service account CSR file ``` cat > service-account-csr.json <<EOF { "CN": "service-accounts", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "US", "L": "Raliegh", "O": "Kubernetes", "OU": "Kubernetes The Hard Way", "ST": "North Carolina" } ] } EOF ``` 2. Generate the signed certificate and key file ``` cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -profile=kubernetes \ service-account-csr.json | cfssljson -bare service-account ``` 3. Verify the service account key pair was created properly. ``` openssl x509 -text -noout -in service-account.pem Certificate: Data: Version: 3 (0x2) Serial Number: ... ``` #### Generated files * `service-account-key.pem` * `service-account.pem` * `service-account-csr.json` * `service-account.csr` That is the last of the 9 certificates and key files you'll need in your kubernetes cluster. In the next section, you'll place them in the proper locations on the proper servers. ### Distributing client and server certificates 1. Distribute the application node certificates and key files ``` for instance in worker-0 worker-1 worker-2; do gcloud compute scp ca.pem ${instance}-key.pem ${instance}.pem ${instance}:~/ done ``` output: ``` Warning: Permanently added 'compute.1224626097461632392' (ECDSA) to the list of known hosts. ca.pem 100% 1338 101.0KB/s 00:00 worker-0-key.pem 100% 1679 125.5KB/s 00:00 worker-0.pem 100% 1505 112.9KB/s 00:00 Warning: Permanently added 'compute.8234720453539634566' (ECDSA) to the list of known hosts. ca.pem 100% 1338 99.8KB/s 00:00 worker-1-key.pem 100% 1675 124.7KB/s 00:00 worker-1.pem 100% 1505 111.7KB/s 00:00 Warning: Permanently added 'compute.4717499042523749764' (ECDSA) to the list of known hosts. ca.pem 100% 1338 96.4KB/s 00:00 worker-2-key.pem 100% 1679 125.3KB/s 00:00 worker-2.pem ``` 2. Distribute the certificates and key files for control plane services. In the next lab you'll use these certificates to create client configuration files. ``` for instance in controller-0 controller-1 controller-2; do gcloud compute scp ca.pem ca-key.pem kubernetes-key.pem kubernetes.pem \ service-account-key.pem service-account.pem ${instance}:~/ done ``` output (*note that controller-0 is already in your known_hosts file*): ``` ca.pem 100% 1338 98.7KB/s 00:00 ca-key.pem 100% 1679 124.1KB/s 00:00 kubernetes-key.pem 100% 1679 124.4KB/s 00:00 kubernetes.pem 100% 1684 125.0KB/s 00:00 service-account-key.pem 100% 1679 124.0KB/s 00:00 service-account.pem 100% 1448 108.6KB/s 00:00 Warning: Permanently added 'compute.557026018878004704' (ECDSA) to the list of known hosts. ca.pem 100% 1338 101.5KB/s 00:00 ca-key.pem 100% 1679 129.4KB/s 00:00 kubernetes-key.pem 100% 1679 128.0KB/s 00:00 kubernetes.pem 100% 1684 129.4KB/s 00:00 service-account-key.pem 100% 1679 128.3KB/s 00:00 service-account.pem 100% 1448 110.2KB/s 00:00 Warning: Permanently added 'compute.1767130885567520254' (ECDSA) to the list of known hosts. ca.pem 100% 1338 99.4KB/s 00:00 ca-key.pem 100% 1679 126.2KB/s 00:00 kubernetes-key.pem 100% 1679 125.5KB/s 00:00 kubernetes.pem 100% 1684 126.8KB/s 00:00 service-account-key.pem 100% 1679 125.9KB/s 00:00 service-account.pem ``` ## Wrap-Up It may feel a little odd to be talking about kubernetes so much while you haven't even downloaded the source code from Github yyout. Don't worry. You'll get there. Think about all of this prep work the next time you use a kube cluster. There's a massive amount of prep work that is normally automated during a kube install. Hopefully this gives you a little more appreciation for that process, as well as a little better understanding of it. If you have any questions, feel free to reach out to jduncan on the [kubernetes slack](https://kubernetes.slack.com/archives/DPXNHT65T) or [twitter](https://twitter.com/jamieeduncan). **Next up in the series is Session Two - https://hackmd.io/RIEmIlXpRouNlVKhBQ3Thw**

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.