# TAP 1.5 baseline
Before running tap install, create these two resources:
1. tap-dev namespace
```
$ kubectl create ns tap-dev
```
2. registry-credentials secret in namespace tap-install
```
```
TAP values.yml
```
shared:
ingress_domain: "taptestazure.carvelind.cloudfocused.in"
image_registry:
project_path: "docker.io/rohitagg2020"
secret:
name: "registry-credentials"
namespace: "tap-install"
kubernetes_version: "1.26.10" # Required regardless of distribution when Kubernetes version is 1.25 or later.
ceip_policy_disclosed: TRUE # Installation fails if this is not set to true. Not a string.
#The above keys are minimum numbers of entries needed in tap-values.yaml to get a functioning TAP Full profile installation.
#Below are the keys which may have default values set, but can be overridden.
profile: full # Can take iterate, build, run, view.
supply_chain: basic # Can take testing, testing_scanning.
ootb_supply_chain_basic: # Based on supply_chain set above, can be changed to ootb_supply_chain_testing, ootb_supply_chain_testing_scanning.
registry:
server: "docker.io/rohitagg2020" # Takes the value from the shared section by default, but can be overridden by setting a different value.
repository: "build-service" # Takes the value from the shared section by default, but can be overridden by setting a different value.
gitops:
ssh_secret: "" # Takes "" as value by default; but can be overridden by setting a different value.
contour:
envoy:
service:
type: LoadBalancer # This is set by default, but can be overridden by setting a different value.
buildservice:
# Takes the value from the shared section by default, but can be overridden by setting a different value.
kp_default_repository: "docker.io/rohitagg2020"
kp_default_repository_secret: # Takes the value from the shared section above by default, but can be overridden by setting a different value.
name: "registry-credentials"
namespace: "tap-install"
tap_gui:
service_type: ClusterIP # If the shared.ingress_domain is set as earlier, this must be set to ClusterIP.
metadataStoreAutoconfiguration: true # Create a service account, the Kubernetes control plane token and the requisite app_config block to enable communications between Tanzu Application Platform GUI and SCST - Store.
app_config:
catalog:
locations:
- type: url
target: https://github.com/kumaritanushree/tap-catalog/blob/master/catalog-info.yaml
metadata_store:
ns_for_export_app_cert: "tap-dev"
app_service_type: ClusterIP # Defaults to LoadBalancer. If shared.ingress_domain is set earlier, this must be set to ClusterIP.
scanning:
metadataStore:
url: "" # Configuration is moved, so set this string to empty.
grype:
namespace: "tap-dev"
# targetImagePullSecret: "TARGET-REGISTRY-CREDENTIALS-SECRET"
# In a single cluster, the connection between the scanning pod and the metadata store happens inside the cluster and does not pass through ingress. This is automatically configured, you do not need to provide an ingress connection to the store.
```
## AWS
* cluster creation: 16m
* install EBS CSI driver: 10m
with error:
```
eksctl create iamserviceaccount \
--name ebs-csi-controller-sa \
--namespace kube-system \
--cluster $EKS_CLUSTER_NAME \
--attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
--approve \
--role-only \
--role-name AmazonEKS_EBS_CSI_DriverRole
2023-03-24 15:05:11 [ℹ] 1 existing iamserviceaccount(s) (kube-system/aws-node) will be excluded
2023-03-24 15:05:11 [ℹ] 1 iamserviceaccount (kube-system/ebs-csi-controller-sa) was included (based on the include/exclude rules)
2023-03-24 15:05:11 [!] serviceaccounts that exist in Kubernetes will be excluded, use --override-existing-serviceaccounts to override
2023-03-24 15:05:11 [ℹ] 1 task: { create IAM role for serviceaccount "kube-system/ebs-csi-controller-sa" }
2023-03-24 15:05:11 [ℹ] building iamserviceaccount stack "eksctl-tap-on-aws-kt-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa"
2023-03-24 15:05:11 [ℹ] deploying stack "eksctl-tap-on-aws-kt-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa"
2023-03-24 15:05:11 [ℹ] waiting for CloudFormation stack "eksctl-tap-on-aws-kt-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa"
2023-03-24 15:05:41 [ℹ] waiting for CloudFormation stack "eksctl-tap-on-aws-kt-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa"
2023-03-24 15:05:41 [ℹ] 1 error(s) occurred and IAM Role stacks haven't been created properly, you may wish to check CloudFormation console
2023-03-24 15:05:41 [✖] waiter state transitioned to Failure
```
* Registry creation and role and arn for that registry: 3m
* cluster essentials install: 4m (including download tar, export var and install. Install took 1m)
* export var for image relocation: 2m
* Image relocation: 1h44m
Need to look into it:
```
ktanushree4XJYT:mumbai-cluster ktanushree$ kubectl get pods -n accelerator-system
NAME READY STATUS RESTARTS AGE
acc-engine-5d596d96dd-mxlv4 1/1 Running 0 4d16h
acc-jobs-28005600-clbhh 0/1 Completed 0 2d2h
acc-jobs-28007040-ksw5c 0/1 Completed 0 26h
acc-jobs-28008480-jhwkj 0/1 Completed 0 167m
acc-server-6f9bf6dfbb-9l7sc 1/1 Running 0 4d16h
accelerator-controller-manager-7dbb6764d4-s9bw7 1/1 Running 0 4d16h
ktanushree4XJYT:mumbai-cluster ktanushree$
ktanushree4XJYT:mumbai-cluster ktanushree$
ktanushree4XJYT:mumbai-cluster ktanushree$
ktanushree4XJYT:mumbai-cluster ktanushree$ kubectl logs acc-jobs-28005600-clbhh
Error from server (NotFound): pods "acc-jobs-28005600-clbhh" not found
ktanushree4XJYT:mumbai-cluster ktanushree$
ktanushree4XJYT:mumbai-cluster ktanushree$ kubectl logs acc-jobs-28005600-clbhh -n accelerator-system
{"level":"info","ts":"2023-04-01T08:00:01.056251634Z","msg":"Starting jobs ..."}
{"level":"info","ts":"2023-04-01T08:00:01.056330825Z","logger":"CleanInvocationEventsJob","msg":"Starting the Job"}
{"level":"info","ts":"2023-04-01T08:00:01.056339338Z","logger":"CleanInvocationEventsJob","msg":"Using","maxAgeInDays":30}
{"level":"info","ts":"2023-04-01T08:00:01.056346905Z","logger":"CleanInvocationEventsJob","msg":"Using","maxAgeDuration":2592000000000000}
{"level":"info","ts":"2023-04-01T08:00:01.064239898Z","msg":"Completed all jobs!"}
```
## AZURE
Cluster Configuration: Configuration: 4 Standard_D4as_v4 nodes, 4 vCPUs and 16 GB RAM each
**Container registry:** 5m (exploring option as well)
details:
name: testtapmages
SKU: Basic (it has 10 GB storage)
location central india
login server: testtapmages.azurecr.io
username: testtapmages
pass: ibICvxj4q9vTY6zEpHUfn4b8c85vmEWaDV+G+wOF9U+ACRDKKyg5
time taken to find username, pass: 20min
(how to get cred: go to registry -> access key-> enable adin user -> copy username and password )
export ENV for image relocation: 4m
finding creds with read, right access for registry: 10m
**image relocation:** 71m
```
export IMGPKG_REGISTRY_HOSTNAME_0=registry.tanzu.vmware.com
export IMGPKG_REGISTRY_USERNAME_0=ktanushree@vmware.com
export IMGPKG_REGISTRY_PASSWORD_0=Tanu@123
export IMGPKG_REGISTRY_HOSTNAME_1=tapimagesnew.azurecr.io
export IMGPKG_REGISTRY_USERNAME_1=tapimagesnew
export IMGPKG_REGISTRY_PASSWORD_1=OOi+cz2mlcqPmLsNgTBohCgt5zATKGpU8hV08msgY2+ACRB+6+Vy
export INSTALL_REGISTRY_USERNAME=tapimagesnew
export INSTALL_REGISTRY_PASSWORD=OOi+cz2mlcqPmLsNgTBohCgt5zATKGpU8hV08msgY2+ACRB+6+Vy
export INSTALL_REGISTRY_HOSTNAME=tapimagesnew.azurecr.io
export TAP_VERSION=1.5.0
export INSTALL_REPO=tap-images
```
**add secrets:**
```
tanzu secret registry add registry-credentials \
--server ${INSTALL_REGISTRY_HOSTNAME} \
--username ${INSTALL_REGISTRY_USERNAME} \
--password ${INSTALL_REGISTRY_PASSWORD} \
--namespace tap-install \
--export-to-all-namespaces \
--yes
```
**cluster creation:** 15m (exploration and some failure due to node unavailability. Actual creation time is 5m)
```
export INSTALL_BUNDLE=registry.tanzu.vmware.com/tanzu-cluster-essentials/cluster-essentials-bundle@sha256:79abddbc3b49b44fc368fede0dab93c266ff7c1fe305e2d555ed52d00361b446
export INSTALL_REGISTRY_HOSTNAME=registry.tanzu.vmware.com
export INSTALL_REGISTRY_USERNAME=ktanushree@vmware.com
export INSTALL_REGISTRY_PASSWORD=Tanu@123
```
**tanzu cli:** 7m (took time to check correct version and cmd)
issues: https://docs-staging.vmware.com/en/draft/VMware-Tanzu-Application-Platform/1.5/tap/install-tanzu-cli.html
tar file name is: tanzu-framework-darwin-amd64-v0.28.1.1.tar
tanzu version is: v0.28.1
for install cmd should be: sudo install cli/core/v0.28.1/tanzu-core-darwin_amd64 /usr/local/bin/tanzu
**Install CE:**
download and unstar: 1m
export env and install: 2m
Add repo and secrets for TAP: 4m (failed for adding secret for registry due to wrong ENVs)
write tap-values.yaml: 15m (go through each field as there are chnages in format)
MY_DEV_NAMESPACE : document should says this namespace should be available before installing TAP.
tap isntall is failing and then started reconciling automatically.
```
$ time tanzu package install tap -p tap.tanzu.vmware.com -v $TAP_VERSION --values-file tap-values.yaml -n tap-install
11:07:52PM: Creating service account 'tap-tap-install-sa'
11:07:52PM: Creating cluster admin role 'tap-tap-install-cluster-role'
11:07:52PM: Creating cluster role binding 'tap-tap-install-cluster-rolebinding'
11:07:52PM: Creating secret 'tap-tap-install-values'
11:07:52PM: Creating overlay secrets
11:07:52PM: Creating package install resource
11:07:53PM: Waiting for PackageInstall reconciliation for 'tap'
11:07:53PM: Fetch started
11:07:53PM: Fetching
| apiVersion: vendir.k14s.io/v1alpha1
| directories:
| - contents:
| - imgpkgBundle:
| image: tapimagesnew.azurecr.io/tap-images/tap-packages@sha256:2ef21f69c3c1d9106836d01d8f0369352a3c30889c41dc8f7455b4b2a8d5c918
| path: .
| path: "0"
| kind: LockConfig
|
11:07:53PM: Fetch succeeded
11:07:54PM: Template succeeded
11:07:54PM: Deploy started (2s ago)
11:07:56PM: Deploying
| Target cluster 'https://10.0.0.1:443' (nodes: aks-agentpool-12860209-vmss000000, 3+)
| Changes
| Namespace Name Kind Age Op Op st. Wait to Rs Ri
| (cluster) tap-install-cluster-admin-role ClusterRole - create - reconcile - -
| ^ tap-install-cluster-admin-role-binding ClusterRoleBinding - create - reconcile - -
| tap-install accelerator PackageInstall - create - reconcile - -
| ^ accelerator-values-ver-1 Secret - create - reconcile - -
| ^ api-auto-registration PackageInstall - create - reconcile - -
| ^ api-auto-registration-values-ver-1 Secret - create - reconcile - -
| ^ api-portal PackageInstall - create - reconcile - -
| ^ api-portal-values-ver-1 Secret - create - reconcile - -
| ^ appliveview PackageInstall - create - reconcile - -
| ^ appliveview-apiserver PackageInstall - create - reconcile - -
| ^ appliveview-apiserver-values-ver-1 Secret - create - reconcile - -
| ^ appliveview-connector PackageInstall - create - reconcile - -
| ^ appliveview-connector-values-ver-1 Secret - create - reconcile - -
| ^ appliveview-conventions PackageInstall - create - reconcile - -
| ^ appliveview-conventions-values-ver-1 Secret - create - reconcile - -
| ^ appliveview-values-ver-1 Secret - create - reconcile - -
| ^ appsso PackageInstall - create - reconcile - -
| ^ appsso-values-ver-1 Secret - create - reconcile - -
| ^ bitnami-services PackageInstall - create - reconcile - -
| ^ bitnami-services-values-ver-1 Secret - create - reconcile - -
| ^ buildservice PackageInstall - create - reconcile - -
| ^ buildservice-values-ver-1 Secret - create - reconcile - -
| ^ cartographer PackageInstall - create - reconcile - -
| ^ cartographer-values-ver-1 Secret - create - reconcile - -
| ^ cert-manager PackageInstall - create - reconcile - -
| ^ cert-manager-values-ver-1 Secret - create - reconcile - -
| ^ cnrs PackageInstall - create - reconcile - -
| ^ cnrs-values-ver-1 Secret - create - reconcile - -
| ^ contour PackageInstall - create - reconcile - -
| ^ contour-values-ver-1 Secret - create - reconcile - -
| ^ crossplane PackageInstall - create - reconcile - -
| ^ crossplane-values-ver-1 Secret - create - reconcile - -
| ^ developer-conventions PackageInstall - create - reconcile - -
| ^ developer-conventions-values-ver-1 Secret - create - reconcile - -
| ^ eventing PackageInstall - create - reconcile - -
| ^ eventing-values-ver-1 Secret - create - reconcile - -
| ^ fluxcd-source-controller PackageInstall - create - reconcile - -
| ^ fluxcd-source-controller-values-ver-1 Secret - create - reconcile - -
| ^ grype PackageInstall - create - reconcile - -
| ^ grype-values-ver-1 Secret - create - reconcile - -
| ^ learningcenter PackageInstall - create - reconcile - -
| ^ learningcenter-values-ver-1 Secret - create - reconcile - -
| ^ learningcenter-workshops PackageInstall - create - reconcile - -
| ^ metadata-store PackageInstall - create - reconcile - -
| ^ metadata-store-values-ver-1 Secret - create - reconcile - -
| ^ namespace-provisioner PackageInstall - create - reconcile - -
| ^ namespace-provisioner-values-ver-1 Secret - create - reconcile - -
| ^ ootb-delivery-basic PackageInstall - create - reconcile - -
| ^ ootb-delivery-basic-values-ver-1 Secret - create - reconcile - -
| ^ ootb-supply-chain-basic PackageInstall - create - reconcile - -
| ^ ootb-supply-chain-basic-values-ver-1 Secret - create - reconcile - -
| ^ ootb-templates PackageInstall - create - reconcile - -
| ^ ootb-templates-values-ver-1 Secret - create - reconcile - -
| ^ policy-controller PackageInstall - create - reconcile - -
| ^ policy-values-ver-1 Secret - create - reconcile - -
| ^ scanning PackageInstall - create - reconcile - -
| ^ scanning-values-ver-1 Secret - create - reconcile - -
| ^ service-bindings PackageInstall - create - reconcile - -
| ^ services-toolkit PackageInstall - create - reconcile - -
| ^ services-toolkit-values-ver-1 Secret - create - reconcile - -
| ^ source-controller PackageInstall - create - reconcile - -
| ^ source-controller-values-ver-1 Secret - create - reconcile - -
| ^ spring-boot-conventions PackageInstall - create - reconcile - -
| ^ springboot-conventions-values-ver-1 Secret - create - reconcile - -
| ^ tap-auth PackageInstall - create - reconcile - -
| ^ tap-gui PackageInstall - create - reconcile - -
| ^ tap-gui-values-ver-1 Secret - create - reconcile - -
| ^ tap-install-sa ServiceAccount - create - reconcile - -
| ^ tap-telemetry PackageInstall - create - reconcile - -
| ^ tap-telemetry-values-ver-1 Secret - create - reconcile - -
| ^ tekton-pipelines PackageInstall - create - reconcile - -
| ^ tekton-pipelines-values-ver-1 Secret - create - reconcile - -
| Op: 72 create, 0 delete, 0 update, 0 noop, 0 exists
| Wait to: 72 reconcile, 0 delete, 0 noop
| 5:37:54PM: ---- applying 35 changes [0/72 done] ----
| 5:37:54PM: create secret/contour-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/source-controller-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/api-auto-registration-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/bitnami-services-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create serviceaccount/tap-install-sa (v1) namespace: tap-install
| 5:37:54PM: create secret/appliveview-apiserver-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/appliveview-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/cert-manager-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/cartographer-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/buildservice-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/ootb-supply-chain-basic-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/crossplane-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/appliveview-conventions-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/cnrs-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/developer-conventions-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/appliveview-connector-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/eventing-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/fluxcd-source-controller-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/learningcenter-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/grype-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/metadata-store-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/namespace-provisioner-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/ootb-delivery-basic-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/springboot-conventions-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/accelerator-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/ootb-templates-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/scanning-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/policy-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/tap-telemetry-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/services-toolkit-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/appsso-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/tap-gui-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/api-portal-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create secret/tekton-pipelines-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: create clusterrole/tap-install-cluster-admin-role (rbac.authorization.k8s.io/v1) cluster
| 5:37:54PM: ---- waiting on 35 changes [0/72 done] ----
| 5:37:54PM: ok: reconcile secret/api-auto-registration-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/source-controller-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile serviceaccount/tap-install-sa (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile clusterrole/tap-install-cluster-admin-role (rbac.authorization.k8s.io/v1) cluster
| 5:37:54PM: ok: reconcile secret/contour-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/appliveview-apiserver-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/cert-manager-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/cartographer-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/buildservice-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/appliveview-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/ootb-supply-chain-basic-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/cnrs-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/appliveview-conventions-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/crossplane-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/developer-conventions-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/grype-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/eventing-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/appliveview-connector-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/fluxcd-source-controller-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/learningcenter-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/ootb-delivery-basic-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/accelerator-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/metadata-store-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/springboot-conventions-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/namespace-provisioner-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/scanning-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/services-toolkit-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/ootb-templates-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/policy-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/tap-telemetry-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/appsso-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/tap-gui-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/bitnami-services-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/api-portal-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ok: reconcile secret/tekton-pipelines-values-ver-1 (v1) namespace: tap-install
| 5:37:54PM: ---- applying 1 changes [35/72 done] ----
| 5:37:54PM: create clusterrolebinding/tap-install-cluster-admin-role-binding (rbac.authorization.k8s.io/v1) cluster
| 5:37:54PM: ---- waiting on 1 changes [35/72 done] ----
| 5:37:54PM: ok: reconcile clusterrolebinding/tap-install-cluster-admin-role-binding (rbac.authorization.k8s.io/v1) cluster
| 5:37:54PM: ---- applying 1 changes [36/72 done] ----
| 5:37:54PM: create packageinstall/tap-telemetry (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:37:54PM: ---- waiting on 1 changes [36/72 done] ----
| 5:37:54PM: ongoing: reconcile packageinstall/tap-telemetry (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:37:54PM: ^ Waiting for generation 1 to be observed
| 5:37:57PM: ongoing: reconcile packageinstall/tap-telemetry (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:37:57PM: ^ Reconciling
| 5:38:03PM: ok: reconcile packageinstall/tap-telemetry (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ---- applying 11 changes [37/72 done] ----
| 5:38:03PM: create packageinstall/cert-manager (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/crossplane (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/appliveview-connector (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/eventing (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/tekton-pipelines (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/buildservice (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/fluxcd-source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/scanning (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/service-bindings (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/namespace-provisioner (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: create packageinstall/tap-auth (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ---- waiting on 11 changes [37/72 done] ----
| 5:38:03PM: ongoing: reconcile packageinstall/cert-manager (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/buildservice (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/service-bindings (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/tap-auth (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/tekton-pipelines (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/crossplane (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/appliveview-connector (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/fluxcd-source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/namespace-provisioner (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/eventing (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:03PM: ongoing: reconcile packageinstall/scanning (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:03PM: ^ Waiting for generation 1 to be observed
| 5:38:06PM: ongoing: reconcile packageinstall/buildservice (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:06PM: ^ Reconciling
| 5:38:06PM: ongoing: reconcile packageinstall/scanning (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:06PM: ^ Reconciling
| 5:38:06PM: ongoing: reconcile packageinstall/cert-manager (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:06PM: ^ Reconciling
| 5:38:06PM: ongoing: reconcile packageinstall/tekton-pipelines (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:06PM: ^ Reconciling
| 5:38:06PM: ongoing: reconcile packageinstall/appliveview-connector (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:06PM: ^ Reconciling
| 5:38:06PM: ongoing: reconcile packageinstall/crossplane (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:06PM: ^ Reconciling
| 5:38:06PM: ongoing: reconcile packageinstall/eventing (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:06PM: ^ Reconciling
| 5:38:09PM: ongoing: reconcile packageinstall/service-bindings (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:09PM: ^ Reconciling
| 5:38:09PM: ongoing: reconcile packageinstall/fluxcd-source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:09PM: ^ Reconciling
| 5:38:09PM: ongoing: reconcile packageinstall/namespace-provisioner (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:09PM: ^ Reconciling
| 5:38:12PM: ongoing: reconcile packageinstall/tap-auth (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:12PM: ^ Reconciling
| 5:38:21PM: ok: reconcile packageinstall/appliveview-connector (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:21PM: ---- waiting on 10 changes [38/72 done] ----
| 5:38:24PM: ok: reconcile packageinstall/fluxcd-source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:24PM: ---- waiting on 9 changes [39/72 done] ----
| 5:38:27PM: ok: reconcile packageinstall/tekton-pipelines (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:27PM: ok: reconcile packageinstall/tap-auth (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:27PM: ---- waiting on 7 changes [41/72 done] ----
| 5:38:30PM: ok: reconcile packageinstall/scanning (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:30PM: ok: reconcile packageinstall/namespace-provisioner (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:30PM: ---- applying 1 changes [48/72 done] ----
| 5:38:30PM: create packageinstall/grype (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:30PM: ---- waiting on 6 changes [43/72 done] ----
| 5:38:30PM: ongoing: reconcile packageinstall/grype (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:30PM: ^ Waiting for generation 1 to be observed
| 5:38:33PM: ok: reconcile packageinstall/service-bindings (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:33PM: ongoing: reconcile packageinstall/grype (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:33PM: ^ Reconciling
| 5:38:33PM: ---- waiting on 5 changes [44/72 done] ----
| 5:38:36PM: ok: reconcile packageinstall/grype (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:36PM: ok: reconcile packageinstall/crossplane (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:36PM: ---- waiting on 3 changes [46/72 done] ----
| 5:38:49PM: ok: reconcile packageinstall/cert-manager (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ---- applying 9 changes [49/72 done] ----
| 5:38:49PM: create packageinstall/appliveview-apiserver (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: create packageinstall/api-auto-registration (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: create packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: create packageinstall/cartographer (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: create packageinstall/appsso (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: create packageinstall/source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: create packageinstall/contour (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: create packageinstall/services-toolkit (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: create packageinstall/policy-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ---- waiting on 11 changes [47/72 done] ----
| 5:38:49PM: ongoing: reconcile packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ^ Waiting for generation 1 to be observed
| 5:38:49PM: ongoing: reconcile packageinstall/cartographer (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ^ Waiting for generation 1 to be observed
| 5:38:49PM: ongoing: reconcile packageinstall/source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ^ Waiting for generation 1 to be observed
| 5:38:49PM: ongoing: reconcile packageinstall/policy-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ^ Waiting for generation 1 to be observed
| 5:38:49PM: ongoing: reconcile packageinstall/appliveview-apiserver (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ^ Waiting for generation 1 to be observed
| 5:38:49PM: ongoing: reconcile packageinstall/appsso (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ^ Waiting for generation 1 to be observed
| 5:38:49PM: ongoing: reconcile packageinstall/api-auto-registration (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ^ Waiting for generation 1 to be observed
| 5:38:49PM: ongoing: reconcile packageinstall/contour (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ^ Waiting for generation 1 to be observed
| 5:38:49PM: ongoing: reconcile packageinstall/services-toolkit (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:49PM: ^ Waiting for generation 1 to be observed
| 5:38:52PM: ongoing: reconcile packageinstall/cartographer (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:52PM: ^ Reconciling
| 5:38:52PM: ongoing: reconcile packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:52PM: ^ Reconciling
| 5:38:52PM: ongoing: reconcile packageinstall/appsso (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:52PM: ^ Reconciling
| 5:38:52PM: ongoing: reconcile packageinstall/appliveview-apiserver (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:52PM: ^ Reconciling
| 5:38:52PM: ongoing: reconcile packageinstall/api-auto-registration (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:52PM: ^ Reconciling
| 5:38:52PM: ongoing: reconcile packageinstall/contour (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:52PM: ^ Reconciling
| 5:38:55PM: ongoing: reconcile packageinstall/policy-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:55PM: ^ Reconciling
| 5:38:55PM: ongoing: reconcile packageinstall/services-toolkit (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:55PM: ^ Reconciling
| 5:38:55PM: ongoing: reconcile packageinstall/source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install
| 5:38:55PM: ^ Reconciling
11:08:58PM: Deploy failed
| kapp: Error: waiting on reconcile packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install:
| Finished unsuccessfully (Reconcile failed: (message: kapp: Error: Expected to find kind 'projectcontour.io/v1/HTTPProxy', but did not:
| - Kubernetes API server did not have matching apiVersion + kind
| - No matching CRD was found in given configuration))
| Deploying: Error (see .status.usefulErrorMessage for details)
11:08:58PM: Error tailing app: Reconciling app: Deploy failed
11:08:59PM: packageinstall/tap (packaging.carvel.dev/v1alpha1) namespace: tap-install: Reconciling
11:09:02PM: packageinstall/tap (packaging.carvel.dev/v1alpha1) namespace: tap-install: ReconcileFailed
Error: packageinstall/tap (packaging.carvel.dev/v1alpha1) namespace: tap-install: Reconciling: kapp: Error: waiting on reconcile packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install:
Finished unsuccessfully (Reconcile failed: (message: kapp: Error: Expected to find kind 'projectcontour.io/v1/HTTPProxy', but did not:
- Kubernetes API server did not have matching apiVersion + kind
- No matching CRD was found in given configuration)). Reconcile failed: Error (see .status.usefulErrorMessage for details)
real 1m10.843s
user 0m0.431s
sys 0m0.218s
```
Could not capture error in api-portal as it started reconcilation automatically in second.
total time taken in TAP installation: 5m
DNS configuration: 5m (used existing DNS zone and need TAP update with new domain)
## AZURE on separate doc
**login to azure and create subscription and resourcegroup:** 1m
**cluster creation:** 4.5m
## TKG
**Image relocation:** 113m (fails 2 times)
```
export IMGPKG_REGISTRY_HOSTNAME_0=registry.tanzu.vmware.com
export IMGPKG_REGISTRY_USERNAME_0=ktanushree@vmware.com
export IMGPKG_REGISTRY_PASSWORD_0=Tanu@123
export IMGPKG_REGISTRY_HOSTNAME_1=harbor-repo.vmware.com
export IMGPKG_REGISTRY_USERNAME_1=ktanushree
export IMGPKG_REGISTRY_PASSWORD_1=Vmw@0222#tanu
export INSTALL_REGISTRY_USERNAME=ktanushree
export INSTALL_REGISTRY_PASSWORD=Vmw@0222#tanu
export INSTALL_REGISTRY_HOSTNAME=harbor-repo.vmware.com
export TAP_VERSION=1.5.0
export INSTALL_REPO=carvelind # repo in harbor for which I have admin permission
```
**Pre-install setup:** env, create secret, add repo -> 1.5m
**create values.yaml:** 5m
**tap install:** 9m (build-service took bit more time to recover after failure)
api-portal always failing with below error and after second it start reconciling automatically (without any change)
```
$ kubectl describe pkgi api-portal -n tap-install
Name: api-portal
Namespace: tap-install
Labels: kapp.k14s.io/app=1681889254528350007
kapp.k14s.io/association=v1.31801edfd2d246e80e8529593f369f83
Annotations: kapp.k14s.io/change-rule: delete before deleting serviceaccount
kapp.k14s.io/change-rule.cert-manager: upsert after upserting cert-manager
kapp.k14s.io/change-rule.telemetry: upsert after upserting telemetry
kapp.k14s.io/identity: v1;tap-install/packaging.carvel.dev/PackageInstall/api-portal;packaging.carvel.dev/v1alpha1
kapp.k14s.io/original:
{"apiVersion":"packaging.carvel.dev/v1alpha1","kind":"PackageInstall","metadata":{"annotations":{"kapp.k14s.io/change-rule":"delete before...
kapp.k14s.io/original-diff-md5: 58e0494c51d30eb3494f7c9198986bb9
API Version: packaging.carvel.dev/v1alpha1
Kind: PackageInstall
Metadata:
Creation Timestamp: 2023-04-19T07:30:51Z
Finalizers:
finalizers.packageinstall.packaging.carvel.dev/delete
Generation: 1
Managed Fields:
API Version: packaging.carvel.dev/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kapp.k14s.io/change-rule:
f:kapp.k14s.io/change-rule.cert-manager:
f:kapp.k14s.io/change-rule.telemetry:
f:kapp.k14s.io/identity:
f:kapp.k14s.io/original:
f:kapp.k14s.io/original-diff-md5:
f:labels:
.:
f:kapp.k14s.io/app:
f:kapp.k14s.io/association:
f:spec:
.:
f:packageRef:
.:
f:refName:
f:versionSelection:
.:
f:constraints:
f:serviceAccountName:
f:values:
Manager: kapp
Operation: Update
Time: 2023-04-19T07:30:51Z
API Version: packaging.carvel.dev/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:finalizers:
.:
v:"finalizers.packageinstall.packaging.carvel.dev/delete":
Manager: kapp-controller
Operation: Update
Time: 2023-04-19T07:30:52Z
API Version: packaging.carvel.dev/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:conditions:
f:friendlyDescription:
f:lastAttemptedVersion:
f:observedGeneration:
f:usefulErrorMessage:
f:version:
Manager: kapp-controller
Operation: Update
Subresource: status
Time: 2023-04-19T07:30:56Z
Resource Version: 311572
UID: 0bec7baa-d8f2-462e-bcda-144564eacd1b
Spec:
Package Ref:
Ref Name: api-portal.tanzu.vmware.com
Version Selection:
Constraints: 1.3.0
Service Account Name: tap-install-sa
Values:
Secret Ref:
Name: api-portal-values-ver-1
Status:
Conditions:
Message: Error (see .status.usefulErrorMessage for details)
Status: True
Type: ReconcileFailed
Friendly Description: Reconcile failed: Error (see .status.usefulErrorMessage for details)
Last Attempted Version: 1.3.0
Observed Generation: 1
Useful Error Message: kapp: Error: Expected to find kind 'projectcontour.io/v1/HTTPProxy', but did not:
- Kubernetes API server did not have matching apiVersion + kind
- No matching CRD was found in given configuration
Version: 1.3.0
Events: <none>
```
**Build service also failed with error and then start reconciling automatically after few seconds:**
```
Status:
Conditions:
Message: Error (see .status.usefulErrorMessage for details)
Status: True
Type: ReconcileFailed
Friendly Description: Reconcile failed: Error (see .status.usefulErrorMessage for details)
Last Attempted Version: 1.10.8
Observed Generation: 1
Useful Error Message: kapp: Error: waiting on reconcile tanzunetdependencyupdater/dependency-updater (buildservice.tanzu.vmware.com/v1alpha1) namespace: build-service:
Finished unsuccessfully (Encountered failure condition Ready == False: CannotImportDescriptor (message: PUT https://harbor-repo.vmware.com/v2/carvelind/build-service/manifests/latest: multiple errors returned: MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:7cc4c1452eb7f0216be80936d9f9255b897332989fecd0489c82bb2f898c5418; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:0057b8952a41a2b717ceb80349588415f9433aa73fd0080911041f4308eb6cc8; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:a775b9fd3da6efc1ca5129ba7791235e2052b990d99e98be23d13da993cd6080; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:e0b9091c372073eac92103f690826fc0c4e8a38c990d1dc3bab51c2f14e4cd99; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:bdbca6a73be9e8b76c1e565ba846d6a6d69e9b82e2e68f2beaacb3dd5b930b2b))
Version: 1.10.8
```
**DNS configuration:** 5m
## Script related rough work
**list of apps name in TAP:**
**Apps Name** -----------------------**Namespace**
1. accelerator -------------------- accelerator-system
1. api-auto-registration ----------- api-auto-registration
1. api-portal ---------------------- api-portal
1. appliveview -------------------- app-live-view
1. appliveview-apiserver appliveview-tokens-system (not sure)
1. appliveview-connector app-live-view-connector
1. appliveview-conventions app-live-view-conventions
1. appsso appsso
1. bitnami-services
1. buildservice build-service
1. cartographer cartographer-system
1. cert-manager cert-manager
1. cnrs
1. contour
3. crossplane
4. developer-conventions developer-conventions
5. eventing
6. fluxcd-source-controller flux-system
7. grype
8. learningcenter
9. learningcenter-workshops
10. metadata-store ------------------ metadata-store
11. namespace-provisioner
12. ootb-delivery-basic
13. ootb-supply-chain-basic
14. ootb-templates
15. policy-controller
16. scanning
17. service-bindings
18. services-toolkit ----------------- services-toolkit
19. source-controller
20. spring-boot-conventions ----------- spring-boot-convention
21. tap ----------------------------- tap-namespace-provisioning
22. tap-auth
23. tap-gui tap-gui
24. tap-telemetry tap-telemetry, vmware-system-telemetry
25. tekton-pipelines tekton-pipelines, tekton-pipelines-resolvers
| Op | Condition | Behavior | Affecting annotations |
|---|---|---|---|
| create | resource present in manifest, but not present in cluster | create resource, see ... | ... |
| update | resource exists in cluster but is different than manifest, for differencing see ... | patch resource | ... |
| delete | resource not present in manifest, but [labelled as owned](./label-ownership) in cluster | delete resource | ... |
| noop | resource in cluster and manifest match, for diff see ... | no change | ... |
| exists | resource exists in manifest and cluster, manifest contents ignored | no change | ... |
## TAP 1.5.1 (to verify fixes)
### Azure
**image relocation:**
```
export IMGPKG_REGISTRY_HOSTNAME_0=registry.tanzu.vmware.com
export IMGPKG_REGISTRY_USERNAME_0=ktanushree@vmware.com
export IMGPKG_REGISTRY_PASSWORD_0=Tanu@123
export IMGPKG_REGISTRY_HOSTNAME_1=tapimages151.azurecr.io
export IMGPKG_REGISTRY_USERNAME_1=tapimages151
export IMGPKG_REGISTRY_PASSWORD_1=SrV/eZQYM8oSlzjRnthcYDBgTs+BD8MbQxyefaHMnM+ACRCNx0hH
export INSTALL_REGISTRY_USERNAME=tapimages151
export INSTALL_REGISTRY_PASSWORD=SrV/eZQYM8oSlzjRnthcYDBgTs+BD8MbQxyefaHMnM+ACRCNx0hH
export INSTALL_REGISTRY_HOSTNAME=tapimages151.azurecr.io
export TAP_VERSION=1.5.1
export INSTALL_REPO=tap-images
```