owned this note
owned this note
Published
Linked with GitHub
# Pre-Req for Demo Appliance for Tanzu Kubernetes Grid 1.2.1 Fling
[TOC]
# Introduction
This workshop takes advantage of the the Demo Appliance for Tanzu Kubernetes Grid (TKG) Fling which bundles all the required dependencies to deploy TKG Clusters running on either VMware Cloud on AWS and/or vSphere 6.7 Update 3 infrastructure.
Please refer to the infrastructure provider specific pre-reqs below before attempting the workshop.
# VMware Cloud on AWS Prerequisite
- [ ] Existing SDDC or deploy new 1-Node SDDC
- [ ] vSphere Management Network (on-prem) or NSX-T Segment (VMC) to run TKG Demo Appliance which has access to VMC vCenter Server
- [ ] DHCP Enabled Network to run TKG Workloads which has access to VMC vCenter Server
- [ ] Desktop to access the SDDC and TKG Demo Appliance with the following ports open
- [ ] Outbound Port 22 (SSH)
- [ ] Outbound Port 443 (vCenter & K8s API)
- [ ] Outbound Port 31001 (K8s Demo App)
## 1. NSX-T Network
### Network Segment
For demo purposes, we will be running both the TKG Demo Appliance and the TKG Management and Workload Cluster on an NSX-T Segment running in VMC.
With TKG 1.2.x, the HAProxy VM has been replaced with [kube-vip](https://kube-vip.io/) and this means that as part of any TKG Cluster deployment (Management or Workload), an additional IP Address must be specified for te Virtual IP.
In our example below, we will carve up our `192.168.2.0/24` in to the following:
| IP | Usage |
| -------- | -------- |
| 192.168.2.1 | Network Gateway |
| 192.168.2.2 | TKG Demo Appliance |
| 192.168.2.3 to 192.168.2.49 | TKG VIP Address Range |
| 192.168.2.50 to 192.168.2.254 | TKG Workload Address Range |
where `192.168.2.3 to 192.168.2.49` will NOT be included in the DHCP scope so that we can manually reserve those for both the TKG Demo Appliance and VIP addressess.
* Create a new Network Segment which will run the TKG workloads with the following configuration:
| Setting | Value |
| ------ | ----------- |
| Segment Name | tkg-network |
| Type | Routed |
| Subents | 192.168.2.1/24 |
Click on the `Save` button to save our initial settings.
> **Note:** If `192.168.2.1/24` is already in use, you can specify another network
* When prompted to edit the new Network Segment, select Yes and then click on `SET DHCP CONFIG` in the upper right hand corner. Toggle the DHCP Config to `Enabled` and then add the DHCP range as specified in the table below and then click on Apply to save the changes.
| Setting | Value |
| ------ | ----------- |
| DHCP Config | Enabled |
| DHCP Ranges | 192.168.2.50-192.168.2.254 |
## 2. NSX-T Inventory Group
### Configure Compute Inventory Group
* Create the following three Inventory Groups for Compute by clicking Add Group and providing the name specified below and then "Set Members" to set the value
| Group Name | Value |
| ------ | ----------- |
| Desktop | IP Address from https://www.whatismyip.com/ or network you will use to connect to TKG Network |
| SDDC Management | The SDDC CIDR used when deploying your SDDC. You can find this under Network & Security->Overview page and search for "Infrastructure Network". It should look like 10.2.0.0/16 as example but subsitute your value |
| TKG Network | 192.168.2.0/24 (or network you used) |
Here is what the Compute Inventory Group should look like after you have completed the above.
### Configure Management Inventory Group
* Create the following two Inventory Groups for Management by clicking Add Group and providing the name specified below and then "Set Members" to the value below
| Group Name | Value |
| ------ | ----------- |
| Desktop | IP Address from https://www.whatismyip.com/ or network you will use to connect to TKG Network |
| TKG Network | 192.168.2.0/24 (or network you used) |
Here is what the Management Inventory Group should look like after you have completed the above.
## 3. NSX-T Edge Gateway Firewall
### Configure Compute Gateway Firewall
* Create the following three Compute Gateway Firewall Rules (ensure to click Publish to actualy create the Firewall Rules)
| Rule Name | Sources | Destinations | Services |
| ------ | ----------- |------ | ----------- |
| Desktop to TKG Network | Desktop | TKG Network | ANY |
| TKG Network to SDDC Management | TKG Network | SDDC Management | Any |
Here is what the Compute Edge Firewall should look like after you have completed the above.
### Configure Management Gateway Firewall
* Create the following Management Gateway Firewall Rules (ensure to click Publish to actualy create the Firewall Rules)
| Rule Name | Sources | Destinations | Services |
| ------ | ----------- |------ | ----------- |
| Desktop to vCenter Server | Desktop | vCenter Server | HTTPS |
| TKG Network to vCenter Server | TKG Network | vCenter Server | HTTPS |
Here is what the Management Edge Firewall should look like after you have completed the above.
## 4. Public IP and NAT (optional for going over internet)
This step is only required if you do not have Desktop system that already has access to the SDDC via Direct Connect and/or VPN.
### Configure Public IP for TKG Demo Appliance
Request a new Public IP Address and name the entry `TKG Demo Appliance`. Make a note of this IP as you will be SSH'ingn to this address during the workshop
### Configure NAT for TKG Demo Appliance
Create a new NAT mapping to the Public IP Address from the previous step using the following settings:
| Name | Public IP | Service | Port | Internal IP |
| ------ | ----------- |------ | ----------- |----------- |
| TKG Demo Appliance | Public IP from prevoius step | Any | Any | 192.168.2.2|(or any valid IP from TKG Network that will be used for TKG Demo Appliance) |
## 5. Configure VMC vCenter Server Inventory
### Configure Resource Pool for TKG in vCenter Server
### Configure VMC VM Folder for TKG in vCenter Server
## 6. Configure VMC vCenter Server Content Library
### Sync K8s and TKG Demo Appliance OVAs
Navigate to **Menu->Content Library** in the vSphere UI and create a new vSphere Content Library with the following configurations:
| Setting | Value |
| -------- | -------- |
| Name | TKG Demo |
| Subscribed URL | https://download3.vmware.com/software/vmw-tools/tkg-demo-appliance/cl4/lib.json |
| Download Content | Immediately |
| Storage | WorkloadDatastore |
> **Note:** Ensure that your vCenter Server has outbound connectivity to sync from the S3 Content Library
To download TKG Demo Appliance offline, you can find it at:
* **TKG Demo Appliance OVA**- https://download3.vmware.com/software/vmw-tools/tkg-demo-appliance/TKG-Demo-Appliance-1.2.1.ova
To download K8s OVA offline, you can also find it on MyVMware:
* **K8s v1.19.3 OVA** - https://my.vmware.com/web/vmware/downloads/details?downloadGroup=TKG-120&productId=988&rPId=53095
* **K8s v1.18.10 OVA** - https://my.vmware.com/web/vmware/downloads/details?downloadGroup=TKG-120&productId=988&rPId=53095
Once the vSphere Content Library has been created, it should start downloading the content immediately.
> **Note:** To verify everything was downloaded correctly, you should see the "Stored Locally" value show *Yes* under the "OVF & OVA Templates" tab of the vSphere Content Library. If you do not see this value, either the content is still being downloaded or you have a connectivity issue preventing you from connecting to the S3 Content Library from the vCenter Server. If you are having issues sync'ing from S3 Content Library in a VMC environment, please make sure you have reviewed the following [accessing S3 endpoint in VMC](https://docs.vmware.com/en/VMware-Cloud-on-AWS/services/com.vmware.vmc-aws-operations/GUID-B501FA3C-EAF9-4005-AC72-155C3F592281.html)
## 7. Configure K8s vSphere Template
### Deploy K8s OVA from vSphere Content Library
Right click on vSphere Content Library item `photon-3-kube-v1.19.3_vmware.1` and select "New VM from this Template" with the following configuration:
| Setting | Value |
| -------- | -------- |
| Name | photon-3-kube-v1.19.3_vmware.1 |
| VM Folder | VM Templates |
| Resource Pool | TKG |
| Storage | WorkloadDatastore |
| Network | tkg-network |
Right click on vSphere Content Library item `photon-3-kube-v1.18.10_vmware.1` and select "New VM from this Template" with the following configuration:
| Setting | Value |
| -------- | -------- |
| Name | photon-3-kube-v1.18.10_vmware.1 |
| VM Folder | VM Templates |
| Resource Pool | TKG |
| Storage | WorkloadDatastore |
| Network | tkg-network |
> **Note:** Do not power on these VMs, these will be used by TKG to provision TKG Clusters
Once both VMs have been deployed, right click on the VM and select `Template->Convert to Template` to convert the VM to vSphere VM Template
## 8. Configure TKG Demo OVA
### Deploy TKG Demo Appliance from vSphere Content Library
Right click on the `TKG` Resource Pool and select "New Virtual Machine" and chose the "Deploy from Template" option and specify the TKG Demo Appliance and provide a name for the VM.
Select the `TKG` VM Folder:
Select the `TKG` Resource Pool:
Select the`WorkloadDatastore` Datastore:
Select `tkg-network` Network:
In the `Networking` section, please fill in the respective setings based on your network configuration. A static IP Address will be required for proper functionality.
Scroll down to `OS Credentials` and select a secure password, especially if you plan to connect from the Internet. SSH key authentication can also be used to connect to appliance. To do so, simply add your SSH key.
Click finish to start the deployment of the OVA.
> **Note:** It is recommended that if you are un-sure of some of the settings to take an offline snapshot prior to powering on, this way you can adjust settings or easily revert the enviorment if you wish to walk through this again without having to re-deploy the OVA
Lastly, power on the TKG Demo Apppliance and then SSH using root to the Public IP that you had requested earlier. If you can access the VM without going over the public internet, then the address would be the IP Address you had configured for the TKG Demo Appliance.
# VMware Cloud on DellEMC Prerequisite
- [ ] Existing SDDC or deploy new 3-Node SDDC (ensure CIDR does not overlap with 172.17.0.0/16 network)
- [ ] NSX-T Segment to run TKG Demo Appliance which will need access to SDDC vCenter Server
- [ ] DHCP Enabled Network to run TKG Workloads which has access to SDDC vCenter Server
- [ ] Desktop access from the upstream customer network to connect to SDDC and TKG Demo Appliance with the following ports open
- [ ] Outbound Port 22 (SSH)
- [ ] Outbound Port 443 (vCenter & K8s API)
- [ ] Outbound Port 31001 (K8s Demo App)
## 1. NSX-T Network
### Network Segment
For demo purposes, we will be running both the TKG Demo Appliance and the TKG Management and Workload Cluster on an NSX-T Segment running in VMConDellEMC.
With TKG 1.2.x, the HAProxy VM has been replaced with [kube-vip](https://kube-vip.io/) and this means that as part of any TKG Cluster deployment (Management or Workload), an additional IP Address must be specified for te Virtual IP.
In our example below, we will use the default NSX-T Segment called `sddc-cgw-network-1` which is configured as`192.168.1.1/24` and we will logically carve it out into the following:
| IP | Usage |
| -------- | -------- |
| 192.168.1.1 | Network Gateway |
| 192.168.1.2 | TKG Demo Appliance |
| 192.168.1.3 to 192.168.1.49 | TKG VIP Address Range |
| 192.168.1.50 to 192.168.1.254 | TKG Workload Address Range |
where `192.168.1.3 to 192.168.1.49` will NOT be included in the DHCP scope so that we can manually reserve those for both the TKG Demo Appliance and VIP addressess.
Click on the three "dots" to edit the sddc-cgw-network-1 and click on the "Edge DHCP COnfig" to change the DHCP scope.
> **Note:** If `192.168.1.1/24` is already in use, you can create another network
## 2. NSX-T Inventory Group
### Configure Compute Inventory Group
* Create the following three Inventory Groups for Compute by clicking Add Group and providing the name specified below and then "Set Members" to set the value
| Group Name | Value |
| ------ | ----------- |
| SDDC Management | The SDDC CIDR used when deploying your SDDC. You can find this under Network & Security->Overview page and search for "Infrastructure Network". It should look like 172.17.5.0/24 as example but subsitute your value |
| TKG Network | 192.168.1.1/24 (or network you used) |
### Configure Management Inventory Group
* Create the following two Inventory Groups for Management by clicking Add Group and providing the name specified below and then "Set Members" to the value below
| Group Name | Value |
| ------ | ----------- |
| TKG Network | 192.168.1.0/24 (or network you used) |
## 3. NSX-T Edge Gateway Firewall
### Configure Compute Gateway Firewall
* Create the following three Compute Gateway Firewall Rules (ensure to click Publish to actualy create the Firewall Rules)
| Rule Name | Sources | Destinations | Services |
| ------ | ----------- |------ | ----------- |
| TKG Network to SDDC Management | TKG Network | SDDC Management | Any |
### Configure Management Gateway Firewall
* Create the following Management Gateway Firewall Rules (ensure to click Publish to actualy create the Firewall Rules)
| Rule Name | Sources | Destinations | Services |
| ------ | ----------- |------ | ----------- |
| TKG Network to vCenter Server | TKG Network | vCenter Server | HTTPS |
## 5. Configure VMConDellEMC vCenter Server Inventory
### Configure Resource Pool for TKG in vCenter Server
### Configure VM Folder for TKG in vCenter Server
## 6. Configure vCenter Server Content Library
### Sync K8s and TKG Demo Appliance OVAs
Navigate to **Menu->Content Library** in the vSphere UI and create a new vSphere Content Library with the following configurations:
| Setting | Value |
| -------- | -------- |
| Name | TKG Demo |
| Subscribed URL | https://download3.vmware.com/software/vmw-tools/tkg-demo-appliance/cl4/lib.json |
| Download Content | Immediately |
| Storage | WorkloadDatastore |
> **Note:** Ensure that your vCenter Server has outbound connectivity to sync from the S3 Content Library
To download TKG Demo Appliance offline, you can find it at:
* **TKG Demo Appliance OVA**- https://download3.vmware.com/software/vmw-tools/tkg-demo-appliance/TKG-Demo-Appliance-1.2.1.ova
To download K8s OVA offline, you can also find it on MyVMware:
* **K8s v1.19.3 OVA** - https://my.vmware.com/web/vmware/downloads/details?downloadGroup=TKG-120&productId=988&rPId=53095
* **K8s v1.18.10 OVA** - https://my.vmware.com/web/vmware/downloads/details?downloadGroup=TKG-120&productId=988&rPId=53095
Once the vSphere Content Library has been created, it should start downloading the content immediately.
> **Note:** To verify everything was downloaded correctly, you should see the "Stored Locally" value show *Yes* under the "OVF & OVA Templates" tab of the vSphere Content Library. If you do not see this value, either the content is still being downloaded or you have a connectivity issue preventing you from connecting to the Content Library from the vCenter Server.
## 7. Configure K8s vSphere Template
### Deploy K8s OVA from vSphere Content Library
Right click on vSphere Content Library item `photon-3-kube-v1.19.3_vmware.2` and select "New VM from this Template" with the following configuration:
| Setting | Value |
| -------- | -------- |
| Name | photon-3-kube-v1.19.3_vmware.2 |
| VM Folder | VM Templates |
| Resource Pool | TKG |
| Storage | WorkloadDatastore |
| Network | tkg-network |
Right click on vSphere Content Library item `photon-3-kube-v1.18.10_vmware.1` and select "New VM from this Template" with the following configuration:
| Setting | Value |
| -------- | -------- |
| Name | photon-3-kube-v1.18.10_vmware.1 |
| VM Folder | VM Templates |
| Resource Pool | TKG |
| Storage | WorkloadDatastore |
| Network | tkg-network |
> **Note:** Do not power on these VMs, these will be used by TKG to provision TKG Clusters
Once both VMs have been deployed, right click on the VM and select `Template->Convert to Template` to convert the VM to vSphere VM Template
## 8. Configure TKG Demo OVA
### Deploy TKG Demo Appliance from vSphere Content Library
Right click on the `TKG` Resource Pool and select "New Virtual Machine" and chose the "Deploy from Template" option and specify the TKG Demo Appliance and provide a name for the VM.
Select the `TKG` VM Folder:
Select the `TKG` Resource Pool:
Select the`WorkloadDatastore` Datastore:
Select `sddc-cgw-network-1` Network:
In the `Networking` section, please fill in the respective setings based on your network configuration. A static IP Address will be required for proper functionality.
Scroll down to `OS Credentials` and select a secure password. SSH key authentication can also be used to connect to appliance. To do so, simply add your SSH key.
Click finish to start the deployment of the OVA.
> **Note:** It is recommended that if you are un-sure of some of the settings to take an offline snapshot prior to powering on, this way you can adjust settings or easily revert the enviorment if you wish to walk through this again without having to re-deploy the OVA
Lastly, power on the TKG Demo Apppliance and then SSH using root and the IP Address (e.g. 192.168.1.2) you had configured for the TKG Demo Appliance.
# vSphere Prerequisite
- [ ] vSphere 6.7 Update 3 environment
- [ ] vSphere Management Network to run TKG Demo Appliance which has access to vCenter Server
- [ ] DHCP Enabled Network to run TKG Workloads which has access to vCenter Server
- [ ] Desktop to access TKG Demo Appliance with the following ports open
- [ ] Outbound Port 22 (SSH)
- [ ] Outbound Port 443 (vCenter & K8s API)
- [ ] Outbound Port 31001 (K8s Demo App)
## 1. vSphere Network
Create a new vSphere Portgroup (Standard/Distributed) or NSX-V/T Network which will run the TKG workloads and can support DHCP. In this example, we will name it `tkg-network` but you can choose any name that you wish.
With TKG 1.2.x, the HAProxy VM has been replaced with [kube-vip](https://kube-vip.io/) and this means that as part of any TKG Cluster deployment (Management or Workload), an additional IP Address must be specified for te Virtual IP.
Here is an example using `192.168.2.0/24` and one way to carve it up:
| IP | Usage |
| -------- | -------- |
| 192.168.2.1 | Network Gateway |
| 192.168.2.2 | TKG Demo Appliance |
| 192.168.2.3 to 192.168.2.49 | TKG VIP Address Range |
| 192.168.2.50 to 192.168.254 | TKG Workload Address Range |
where `192.168.2.3 to 192.168.2.49` will NOT be included in the DHCP scope so that we can manually reserve those for both the TKG Demo Appliance and VIP addressess.
## 2. Configure vCenter Server Inventory
### Configure Resource Pool for TKG in vCenter Server
### Configure VM Folder for TKG in vCenter Server
## 3. Configure vCenter Server Content Library
### Sync K8s and TKG Demo Appliance OVAs
Navigate to **Menu->Content Library** in the vSphere UI and create a new vSphere Content Library with the following configurations:
| Setting | Value |
| -------- | -------- |
| Name | TKG Demo |
| Subscribed URL | https://download3.vmware.com/software/vmw-tools/tkg-demo-appliance/cl4/lib.json |
| Download Content | Immediately |
| Storage | Select vSphere Datastore |
> **Note:** Ensure that your vCenter Server has outbound connectivity to sync from the S3 Content Library
To download TKG Demo Appliance offline, you can find it at:
* **TKG Demo Appliance OVA**- https://download3.vmware.com/software/vmw-tools/tkg-demo-appliance/TKG-Demo-Appliance-1.2.1.ova
To download K8s and HA Proxy OVA offline, you can also find it on MyVMware:
* **K8s v1.19.3 OVA** - https://my.vmware.com/web/vmware/downloads/details?downloadGroup=TKG-120&productId=988&rPId=53095
* **K8s v1.18.10 OVA** - https://my.vmware.com/web/vmware/downloads/details?downloadGroup=TKG-120&productId=988&rPId=53095
Once the vSphere Content Library has been created, it should start downloading the content immediately.
> **Note:** To verify everything was downloaded correctly, you should see the "Stored Locally" value show *Yes* under the "OVF & OVA Templates" tab of the vSphere Content Library. If you do not see this value, either the content is still being downloaded or you have a connectivity issue preventing you from connecting to the S3 Content Library from the vCenter Server.
## 4. Configure K8s vSphere Template
### Deploy K8s vSphere Content Library
Right click on vSphere Content Library item `photon-3-kube-v1.19.3_vmware.2` and select "New VM from this Template" with the following configuration:
| Setting | Value |
| -------- | -------- |
| Name | photon-3-kube-v1.19.3_vmware.2 |
| VM Folder | VM Templates |
| Resource Pool | TKG |
| Storage | WorkloadDatastore |
| Network | tkg-network |
Right click on vSphere Content Library item `photon-3-kube-v1.18.10_vmware.1` and select "New VM from this Template" with the following configuration:
| Setting | Value |
| -------- | -------- |
| Name | photon-3-kube-v1.18.10_vmware.1 |
| VM Folder | VM Templates |
| Resource Pool | TKG |
| Storage | WorkloadDatastore |
| Network | tkg-network |
> **Note:** Do not power on these VMs, these will be used by TKG to provision TKG Clusters
Once both VMs have been deployed, right click on the VM and select "Template->Convert to Template" to convert the VM to vSphere VM Template
## 5. Configure TKG Demo Appliance
### Deploy TKG Demo Appliance from vSphere Content Library
Right click on the `TKG` Resource Pool and select "New Virtual Machine" and chose the "Deploy from Template" option and specify the TKG Demo Appliance and provide a name for the VM.
Select the `TKG` VM Folder:
Select the `TKG` Resource Pool:
Select your vSphere Datastore:
Select `tkg-network` Network:
In the `Networking` section, please fill in the respective setings based on your network configuration. A static IP Address will be required for proper functionality.
Scroll down to `OS Credentials` and select a secure password, especially if you plan to connect from the Internet. SSH key authentication can also be used to connect to appliance. To do so, simply add your SSH key.
Click finish to start the deployment of the OVA.
> **Note:** It is recommended that if you are un-sure of some of the settings to take an offline snapshot prior to powering on, this way you can adjust settings or easily revert the enviorment if you wish to walk through this again without having to re-deploy the OVA
Finally, right click on the TKG Demo Appliance VM and power it on
Lastly, power on the TKG Demo Apppliance and then SSH using root the IP Address you had configured for the TKG Demo Appliance.
