# TAP 1.5 baseline Before running tap install, create these two resources: 1. tap-dev namespace ``` $ kubectl create ns tap-dev ``` 2. registry-credentials secret in namespace tap-install ``` ``` TAP values.yml ``` shared: ingress_domain: "taptestazure.carvelind.cloudfocused.in" image_registry: project_path: "docker.io/rohitagg2020" secret: name: "registry-credentials" namespace: "tap-install" kubernetes_version: "1.26.10" # Required regardless of distribution when Kubernetes version is 1.25 or later. ceip_policy_disclosed: TRUE # Installation fails if this is not set to true. Not a string. #The above keys are minimum numbers of entries needed in tap-values.yaml to get a functioning TAP Full profile installation. #Below are the keys which may have default values set, but can be overridden. profile: full # Can take iterate, build, run, view. supply_chain: basic # Can take testing, testing_scanning. ootb_supply_chain_basic: # Based on supply_chain set above, can be changed to ootb_supply_chain_testing, ootb_supply_chain_testing_scanning. registry: server: "docker.io/rohitagg2020" # Takes the value from the shared section by default, but can be overridden by setting a different value. repository: "build-service" # Takes the value from the shared section by default, but can be overridden by setting a different value. gitops: ssh_secret: "" # Takes "" as value by default; but can be overridden by setting a different value. contour: envoy: service: type: LoadBalancer # This is set by default, but can be overridden by setting a different value. buildservice: # Takes the value from the shared section by default, but can be overridden by setting a different value. kp_default_repository: "docker.io/rohitagg2020" kp_default_repository_secret: # Takes the value from the shared section above by default, but can be overridden by setting a different value. name: "registry-credentials" namespace: "tap-install" tap_gui: service_type: ClusterIP # If the shared.ingress_domain is set as earlier, this must be set to ClusterIP. metadataStoreAutoconfiguration: true # Create a service account, the Kubernetes control plane token and the requisite app_config block to enable communications between Tanzu Application Platform GUI and SCST - Store. app_config: catalog: locations: - type: url target: https://github.com/kumaritanushree/tap-catalog/blob/master/catalog-info.yaml metadata_store: ns_for_export_app_cert: "tap-dev" app_service_type: ClusterIP # Defaults to LoadBalancer. If shared.ingress_domain is set earlier, this must be set to ClusterIP. scanning: metadataStore: url: "" # Configuration is moved, so set this string to empty. grype: namespace: "tap-dev" # targetImagePullSecret: "TARGET-REGISTRY-CREDENTIALS-SECRET" # In a single cluster, the connection between the scanning pod and the metadata store happens inside the cluster and does not pass through ingress. This is automatically configured, you do not need to provide an ingress connection to the store. ``` ## AWS * cluster creation: 16m * install EBS CSI driver: 10m with error: ``` eksctl create iamserviceaccount \ --name ebs-csi-controller-sa \ --namespace kube-system \ --cluster $EKS_CLUSTER_NAME \ --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \ --approve \ --role-only \ --role-name AmazonEKS_EBS_CSI_DriverRole 2023-03-24 15:05:11 [ℹ] 1 existing iamserviceaccount(s) (kube-system/aws-node) will be excluded 2023-03-24 15:05:11 [ℹ] 1 iamserviceaccount (kube-system/ebs-csi-controller-sa) was included (based on the include/exclude rules) 2023-03-24 15:05:11 [!] serviceaccounts that exist in Kubernetes will be excluded, use --override-existing-serviceaccounts to override 2023-03-24 15:05:11 [ℹ] 1 task: { create IAM role for serviceaccount "kube-system/ebs-csi-controller-sa" } 2023-03-24 15:05:11 [ℹ] building iamserviceaccount stack "eksctl-tap-on-aws-kt-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa" 2023-03-24 15:05:11 [ℹ] deploying stack "eksctl-tap-on-aws-kt-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa" 2023-03-24 15:05:11 [ℹ] waiting for CloudFormation stack "eksctl-tap-on-aws-kt-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa" 2023-03-24 15:05:41 [ℹ] waiting for CloudFormation stack "eksctl-tap-on-aws-kt-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa" 2023-03-24 15:05:41 [ℹ] 1 error(s) occurred and IAM Role stacks haven't been created properly, you may wish to check CloudFormation console 2023-03-24 15:05:41 [✖] waiter state transitioned to Failure ``` * Registry creation and role and arn for that registry: 3m * cluster essentials install: 4m (including download tar, export var and install. Install took 1m) * export var for image relocation: 2m * Image relocation: 1h44m Need to look into it: ``` ktanushree4XJYT:mumbai-cluster ktanushree$ kubectl get pods -n accelerator-system NAME READY STATUS RESTARTS AGE acc-engine-5d596d96dd-mxlv4 1/1 Running 0 4d16h acc-jobs-28005600-clbhh 0/1 Completed 0 2d2h acc-jobs-28007040-ksw5c 0/1 Completed 0 26h acc-jobs-28008480-jhwkj 0/1 Completed 0 167m acc-server-6f9bf6dfbb-9l7sc 1/1 Running 0 4d16h accelerator-controller-manager-7dbb6764d4-s9bw7 1/1 Running 0 4d16h ktanushree4XJYT:mumbai-cluster ktanushree$ ktanushree4XJYT:mumbai-cluster ktanushree$ ktanushree4XJYT:mumbai-cluster ktanushree$ ktanushree4XJYT:mumbai-cluster ktanushree$ kubectl logs acc-jobs-28005600-clbhh Error from server (NotFound): pods "acc-jobs-28005600-clbhh" not found ktanushree4XJYT:mumbai-cluster ktanushree$ ktanushree4XJYT:mumbai-cluster ktanushree$ kubectl logs acc-jobs-28005600-clbhh -n accelerator-system {"level":"info","ts":"2023-04-01T08:00:01.056251634Z","msg":"Starting jobs ..."} {"level":"info","ts":"2023-04-01T08:00:01.056330825Z","logger":"CleanInvocationEventsJob","msg":"Starting the Job"} {"level":"info","ts":"2023-04-01T08:00:01.056339338Z","logger":"CleanInvocationEventsJob","msg":"Using","maxAgeInDays":30} {"level":"info","ts":"2023-04-01T08:00:01.056346905Z","logger":"CleanInvocationEventsJob","msg":"Using","maxAgeDuration":2592000000000000} {"level":"info","ts":"2023-04-01T08:00:01.064239898Z","msg":"Completed all jobs!"} ``` ## AZURE Cluster Configuration: Configuration: 4 Standard_D4as_v4 nodes, 4 vCPUs and 16 GB RAM each **Container registry:** 5m (exploring option as well) details: name: testtapmages SKU: Basic (it has 10 GB storage) location central india login server: testtapmages.azurecr.io username: testtapmages pass: ibICvxj4q9vTY6zEpHUfn4b8c85vmEWaDV+G+wOF9U+ACRDKKyg5 time taken to find username, pass: 20min (how to get cred: go to registry -> access key-> enable adin user -> copy username and password ) export ENV for image relocation: 4m finding creds with read, right access for registry: 10m **image relocation:** 71m ``` export IMGPKG_REGISTRY_HOSTNAME_0=registry.tanzu.vmware.com export IMGPKG_REGISTRY_USERNAME_0=ktanushree@vmware.com export IMGPKG_REGISTRY_PASSWORD_0=Tanu@123 export IMGPKG_REGISTRY_HOSTNAME_1=tapimagesnew.azurecr.io export IMGPKG_REGISTRY_USERNAME_1=tapimagesnew export IMGPKG_REGISTRY_PASSWORD_1=OOi+cz2mlcqPmLsNgTBohCgt5zATKGpU8hV08msgY2+ACRB+6+Vy export INSTALL_REGISTRY_USERNAME=tapimagesnew export INSTALL_REGISTRY_PASSWORD=OOi+cz2mlcqPmLsNgTBohCgt5zATKGpU8hV08msgY2+ACRB+6+Vy export INSTALL_REGISTRY_HOSTNAME=tapimagesnew.azurecr.io export TAP_VERSION=1.5.0 export INSTALL_REPO=tap-images ``` **add secrets:** ``` tanzu secret registry add registry-credentials \ --server ${INSTALL_REGISTRY_HOSTNAME} \ --username ${INSTALL_REGISTRY_USERNAME} \ --password ${INSTALL_REGISTRY_PASSWORD} \ --namespace tap-install \ --export-to-all-namespaces \ --yes ``` **cluster creation:** 15m (exploration and some failure due to node unavailability. Actual creation time is 5m) ``` export INSTALL_BUNDLE=registry.tanzu.vmware.com/tanzu-cluster-essentials/cluster-essentials-bundle@sha256:79abddbc3b49b44fc368fede0dab93c266ff7c1fe305e2d555ed52d00361b446 export INSTALL_REGISTRY_HOSTNAME=registry.tanzu.vmware.com export INSTALL_REGISTRY_USERNAME=ktanushree@vmware.com export INSTALL_REGISTRY_PASSWORD=Tanu@123 ``` **tanzu cli:** 7m (took time to check correct version and cmd) issues: https://docs-staging.vmware.com/en/draft/VMware-Tanzu-Application-Platform/1.5/tap/install-tanzu-cli.html tar file name is: tanzu-framework-darwin-amd64-v0.28.1.1.tar tanzu version is: v0.28.1 for install cmd should be: sudo install cli/core/v0.28.1/tanzu-core-darwin_amd64 /usr/local/bin/tanzu **Install CE:** download and unstar: 1m export env and install: 2m Add repo and secrets for TAP: 4m (failed for adding secret for registry due to wrong ENVs) write tap-values.yaml: 15m (go through each field as there are chnages in format) MY_DEV_NAMESPACE : document should says this namespace should be available before installing TAP. tap isntall is failing and then started reconciling automatically. ``` $ time tanzu package install tap -p tap.tanzu.vmware.com -v $TAP_VERSION --values-file tap-values.yaml -n tap-install 11:07:52PM: Creating service account 'tap-tap-install-sa' 11:07:52PM: Creating cluster admin role 'tap-tap-install-cluster-role' 11:07:52PM: Creating cluster role binding 'tap-tap-install-cluster-rolebinding' 11:07:52PM: Creating secret 'tap-tap-install-values' 11:07:52PM: Creating overlay secrets 11:07:52PM: Creating package install resource 11:07:53PM: Waiting for PackageInstall reconciliation for 'tap' 11:07:53PM: Fetch started 11:07:53PM: Fetching | apiVersion: vendir.k14s.io/v1alpha1 | directories: | - contents: | - imgpkgBundle: | image: tapimagesnew.azurecr.io/tap-images/tap-packages@sha256:2ef21f69c3c1d9106836d01d8f0369352a3c30889c41dc8f7455b4b2a8d5c918 | path: . | path: "0" | kind: LockConfig | 11:07:53PM: Fetch succeeded 11:07:54PM: Template succeeded 11:07:54PM: Deploy started (2s ago) 11:07:56PM: Deploying | Target cluster 'https://10.0.0.1:443' (nodes: aks-agentpool-12860209-vmss000000, 3+) | Changes | Namespace Name Kind Age Op Op st. Wait to Rs Ri | (cluster) tap-install-cluster-admin-role ClusterRole - create - reconcile - - | ^ tap-install-cluster-admin-role-binding ClusterRoleBinding - create - reconcile - - | tap-install accelerator PackageInstall - create - reconcile - - | ^ accelerator-values-ver-1 Secret - create - reconcile - - | ^ api-auto-registration PackageInstall - create - reconcile - - | ^ api-auto-registration-values-ver-1 Secret - create - reconcile - - | ^ api-portal PackageInstall - create - reconcile - - | ^ api-portal-values-ver-1 Secret - create - reconcile - - | ^ appliveview PackageInstall - create - reconcile - - | ^ appliveview-apiserver PackageInstall - create - reconcile - - | ^ appliveview-apiserver-values-ver-1 Secret - create - reconcile - - | ^ appliveview-connector PackageInstall - create - reconcile - - | ^ appliveview-connector-values-ver-1 Secret - create - reconcile - - | ^ appliveview-conventions PackageInstall - create - reconcile - - | ^ appliveview-conventions-values-ver-1 Secret - create - reconcile - - | ^ appliveview-values-ver-1 Secret - create - reconcile - - | ^ appsso PackageInstall - create - reconcile - - | ^ appsso-values-ver-1 Secret - create - reconcile - - | ^ bitnami-services PackageInstall - create - reconcile - - | ^ bitnami-services-values-ver-1 Secret - create - reconcile - - | ^ buildservice PackageInstall - create - reconcile - - | ^ buildservice-values-ver-1 Secret - create - reconcile - - | ^ cartographer PackageInstall - create - reconcile - - | ^ cartographer-values-ver-1 Secret - create - reconcile - - | ^ cert-manager PackageInstall - create - reconcile - - | ^ cert-manager-values-ver-1 Secret - create - reconcile - - | ^ cnrs PackageInstall - create - reconcile - - | ^ cnrs-values-ver-1 Secret - create - reconcile - - | ^ contour PackageInstall - create - reconcile - - | ^ contour-values-ver-1 Secret - create - reconcile - - | ^ crossplane PackageInstall - create - reconcile - - | ^ crossplane-values-ver-1 Secret - create - reconcile - - | ^ developer-conventions PackageInstall - create - reconcile - - | ^ developer-conventions-values-ver-1 Secret - create - reconcile - - | ^ eventing PackageInstall - create - reconcile - - | ^ eventing-values-ver-1 Secret - create - reconcile - - | ^ fluxcd-source-controller PackageInstall - create - reconcile - - | ^ fluxcd-source-controller-values-ver-1 Secret - create - reconcile - - | ^ grype PackageInstall - create - reconcile - - | ^ grype-values-ver-1 Secret - create - reconcile - - | ^ learningcenter PackageInstall - create - reconcile - - | ^ learningcenter-values-ver-1 Secret - create - reconcile - - | ^ learningcenter-workshops PackageInstall - create - reconcile - - | ^ metadata-store PackageInstall - create - reconcile - - | ^ metadata-store-values-ver-1 Secret - create - reconcile - - | ^ namespace-provisioner PackageInstall - create - reconcile - - | ^ namespace-provisioner-values-ver-1 Secret - create - reconcile - - | ^ ootb-delivery-basic PackageInstall - create - reconcile - - | ^ ootb-delivery-basic-values-ver-1 Secret - create - reconcile - - | ^ ootb-supply-chain-basic PackageInstall - create - reconcile - - | ^ ootb-supply-chain-basic-values-ver-1 Secret - create - reconcile - - | ^ ootb-templates PackageInstall - create - reconcile - - | ^ ootb-templates-values-ver-1 Secret - create - reconcile - - | ^ policy-controller PackageInstall - create - reconcile - - | ^ policy-values-ver-1 Secret - create - reconcile - - | ^ scanning PackageInstall - create - reconcile - - | ^ scanning-values-ver-1 Secret - create - reconcile - - | ^ service-bindings PackageInstall - create - reconcile - - | ^ services-toolkit PackageInstall - create - reconcile - - | ^ services-toolkit-values-ver-1 Secret - create - reconcile - - | ^ source-controller PackageInstall - create - reconcile - - | ^ source-controller-values-ver-1 Secret - create - reconcile - - | ^ spring-boot-conventions PackageInstall - create - reconcile - - | ^ springboot-conventions-values-ver-1 Secret - create - reconcile - - | ^ tap-auth PackageInstall - create - reconcile - - | ^ tap-gui PackageInstall - create - reconcile - - | ^ tap-gui-values-ver-1 Secret - create - reconcile - - | ^ tap-install-sa ServiceAccount - create - reconcile - - | ^ tap-telemetry PackageInstall - create - reconcile - - | ^ tap-telemetry-values-ver-1 Secret - create - reconcile - - | ^ tekton-pipelines PackageInstall - create - reconcile - - | ^ tekton-pipelines-values-ver-1 Secret - create - reconcile - - | Op: 72 create, 0 delete, 0 update, 0 noop, 0 exists | Wait to: 72 reconcile, 0 delete, 0 noop | 5:37:54PM: ---- applying 35 changes [0/72 done] ---- | 5:37:54PM: create secret/contour-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/source-controller-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/api-auto-registration-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/bitnami-services-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create serviceaccount/tap-install-sa (v1) namespace: tap-install | 5:37:54PM: create secret/appliveview-apiserver-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/appliveview-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/cert-manager-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/cartographer-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/buildservice-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/ootb-supply-chain-basic-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/crossplane-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/appliveview-conventions-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/cnrs-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/developer-conventions-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/appliveview-connector-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/eventing-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/fluxcd-source-controller-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/learningcenter-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/grype-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/metadata-store-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/namespace-provisioner-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/ootb-delivery-basic-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/springboot-conventions-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/accelerator-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/ootb-templates-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/scanning-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/policy-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/tap-telemetry-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/services-toolkit-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/appsso-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/tap-gui-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/api-portal-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create secret/tekton-pipelines-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: create clusterrole/tap-install-cluster-admin-role (rbac.authorization.k8s.io/v1) cluster | 5:37:54PM: ---- waiting on 35 changes [0/72 done] ---- | 5:37:54PM: ok: reconcile secret/api-auto-registration-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/source-controller-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile serviceaccount/tap-install-sa (v1) namespace: tap-install | 5:37:54PM: ok: reconcile clusterrole/tap-install-cluster-admin-role (rbac.authorization.k8s.io/v1) cluster | 5:37:54PM: ok: reconcile secret/contour-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/appliveview-apiserver-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/cert-manager-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/cartographer-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/buildservice-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/appliveview-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/ootb-supply-chain-basic-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/cnrs-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/appliveview-conventions-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/crossplane-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/developer-conventions-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/grype-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/eventing-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/appliveview-connector-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/fluxcd-source-controller-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/learningcenter-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/ootb-delivery-basic-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/accelerator-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/metadata-store-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/springboot-conventions-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/namespace-provisioner-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/scanning-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/services-toolkit-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/ootb-templates-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/policy-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/tap-telemetry-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/appsso-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/tap-gui-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/bitnami-services-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/api-portal-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ok: reconcile secret/tekton-pipelines-values-ver-1 (v1) namespace: tap-install | 5:37:54PM: ---- applying 1 changes [35/72 done] ---- | 5:37:54PM: create clusterrolebinding/tap-install-cluster-admin-role-binding (rbac.authorization.k8s.io/v1) cluster | 5:37:54PM: ---- waiting on 1 changes [35/72 done] ---- | 5:37:54PM: ok: reconcile clusterrolebinding/tap-install-cluster-admin-role-binding (rbac.authorization.k8s.io/v1) cluster | 5:37:54PM: ---- applying 1 changes [36/72 done] ---- | 5:37:54PM: create packageinstall/tap-telemetry (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:37:54PM: ---- waiting on 1 changes [36/72 done] ---- | 5:37:54PM: ongoing: reconcile packageinstall/tap-telemetry (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:37:54PM: ^ Waiting for generation 1 to be observed | 5:37:57PM: ongoing: reconcile packageinstall/tap-telemetry (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:37:57PM: ^ Reconciling | 5:38:03PM: ok: reconcile packageinstall/tap-telemetry (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ---- applying 11 changes [37/72 done] ---- | 5:38:03PM: create packageinstall/cert-manager (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/crossplane (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/appliveview-connector (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/eventing (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/tekton-pipelines (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/buildservice (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/fluxcd-source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/scanning (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/service-bindings (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/namespace-provisioner (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: create packageinstall/tap-auth (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ---- waiting on 11 changes [37/72 done] ---- | 5:38:03PM: ongoing: reconcile packageinstall/cert-manager (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/buildservice (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/service-bindings (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/tap-auth (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/tekton-pipelines (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/crossplane (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/appliveview-connector (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/fluxcd-source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/namespace-provisioner (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/eventing (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:03PM: ongoing: reconcile packageinstall/scanning (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:03PM: ^ Waiting for generation 1 to be observed | 5:38:06PM: ongoing: reconcile packageinstall/buildservice (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:06PM: ^ Reconciling | 5:38:06PM: ongoing: reconcile packageinstall/scanning (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:06PM: ^ Reconciling | 5:38:06PM: ongoing: reconcile packageinstall/cert-manager (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:06PM: ^ Reconciling | 5:38:06PM: ongoing: reconcile packageinstall/tekton-pipelines (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:06PM: ^ Reconciling | 5:38:06PM: ongoing: reconcile packageinstall/appliveview-connector (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:06PM: ^ Reconciling | 5:38:06PM: ongoing: reconcile packageinstall/crossplane (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:06PM: ^ Reconciling | 5:38:06PM: ongoing: reconcile packageinstall/eventing (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:06PM: ^ Reconciling | 5:38:09PM: ongoing: reconcile packageinstall/service-bindings (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:09PM: ^ Reconciling | 5:38:09PM: ongoing: reconcile packageinstall/fluxcd-source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:09PM: ^ Reconciling | 5:38:09PM: ongoing: reconcile packageinstall/namespace-provisioner (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:09PM: ^ Reconciling | 5:38:12PM: ongoing: reconcile packageinstall/tap-auth (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:12PM: ^ Reconciling | 5:38:21PM: ok: reconcile packageinstall/appliveview-connector (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:21PM: ---- waiting on 10 changes [38/72 done] ---- | 5:38:24PM: ok: reconcile packageinstall/fluxcd-source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:24PM: ---- waiting on 9 changes [39/72 done] ---- | 5:38:27PM: ok: reconcile packageinstall/tekton-pipelines (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:27PM: ok: reconcile packageinstall/tap-auth (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:27PM: ---- waiting on 7 changes [41/72 done] ---- | 5:38:30PM: ok: reconcile packageinstall/scanning (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:30PM: ok: reconcile packageinstall/namespace-provisioner (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:30PM: ---- applying 1 changes [48/72 done] ---- | 5:38:30PM: create packageinstall/grype (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:30PM: ---- waiting on 6 changes [43/72 done] ---- | 5:38:30PM: ongoing: reconcile packageinstall/grype (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:30PM: ^ Waiting for generation 1 to be observed | 5:38:33PM: ok: reconcile packageinstall/service-bindings (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:33PM: ongoing: reconcile packageinstall/grype (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:33PM: ^ Reconciling | 5:38:33PM: ---- waiting on 5 changes [44/72 done] ---- | 5:38:36PM: ok: reconcile packageinstall/grype (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:36PM: ok: reconcile packageinstall/crossplane (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:36PM: ---- waiting on 3 changes [46/72 done] ---- | 5:38:49PM: ok: reconcile packageinstall/cert-manager (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ---- applying 9 changes [49/72 done] ---- | 5:38:49PM: create packageinstall/appliveview-apiserver (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: create packageinstall/api-auto-registration (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: create packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: create packageinstall/cartographer (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: create packageinstall/appsso (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: create packageinstall/source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: create packageinstall/contour (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: create packageinstall/services-toolkit (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: create packageinstall/policy-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ---- waiting on 11 changes [47/72 done] ---- | 5:38:49PM: ongoing: reconcile packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ^ Waiting for generation 1 to be observed | 5:38:49PM: ongoing: reconcile packageinstall/cartographer (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ^ Waiting for generation 1 to be observed | 5:38:49PM: ongoing: reconcile packageinstall/source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ^ Waiting for generation 1 to be observed | 5:38:49PM: ongoing: reconcile packageinstall/policy-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ^ Waiting for generation 1 to be observed | 5:38:49PM: ongoing: reconcile packageinstall/appliveview-apiserver (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ^ Waiting for generation 1 to be observed | 5:38:49PM: ongoing: reconcile packageinstall/appsso (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ^ Waiting for generation 1 to be observed | 5:38:49PM: ongoing: reconcile packageinstall/api-auto-registration (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ^ Waiting for generation 1 to be observed | 5:38:49PM: ongoing: reconcile packageinstall/contour (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ^ Waiting for generation 1 to be observed | 5:38:49PM: ongoing: reconcile packageinstall/services-toolkit (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:49PM: ^ Waiting for generation 1 to be observed | 5:38:52PM: ongoing: reconcile packageinstall/cartographer (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:52PM: ^ Reconciling | 5:38:52PM: ongoing: reconcile packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:52PM: ^ Reconciling | 5:38:52PM: ongoing: reconcile packageinstall/appsso (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:52PM: ^ Reconciling | 5:38:52PM: ongoing: reconcile packageinstall/appliveview-apiserver (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:52PM: ^ Reconciling | 5:38:52PM: ongoing: reconcile packageinstall/api-auto-registration (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:52PM: ^ Reconciling | 5:38:52PM: ongoing: reconcile packageinstall/contour (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:52PM: ^ Reconciling | 5:38:55PM: ongoing: reconcile packageinstall/policy-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:55PM: ^ Reconciling | 5:38:55PM: ongoing: reconcile packageinstall/services-toolkit (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:55PM: ^ Reconciling | 5:38:55PM: ongoing: reconcile packageinstall/source-controller (packaging.carvel.dev/v1alpha1) namespace: tap-install | 5:38:55PM: ^ Reconciling 11:08:58PM: Deploy failed | kapp: Error: waiting on reconcile packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install: | Finished unsuccessfully (Reconcile failed: (message: kapp: Error: Expected to find kind 'projectcontour.io/v1/HTTPProxy', but did not: | - Kubernetes API server did not have matching apiVersion + kind | - No matching CRD was found in given configuration)) | Deploying: Error (see .status.usefulErrorMessage for details) 11:08:58PM: Error tailing app: Reconciling app: Deploy failed 11:08:59PM: packageinstall/tap (packaging.carvel.dev/v1alpha1) namespace: tap-install: Reconciling 11:09:02PM: packageinstall/tap (packaging.carvel.dev/v1alpha1) namespace: tap-install: ReconcileFailed Error: packageinstall/tap (packaging.carvel.dev/v1alpha1) namespace: tap-install: Reconciling: kapp: Error: waiting on reconcile packageinstall/api-portal (packaging.carvel.dev/v1alpha1) namespace: tap-install: Finished unsuccessfully (Reconcile failed: (message: kapp: Error: Expected to find kind 'projectcontour.io/v1/HTTPProxy', but did not: - Kubernetes API server did not have matching apiVersion + kind - No matching CRD was found in given configuration)). Reconcile failed: Error (see .status.usefulErrorMessage for details) real 1m10.843s user 0m0.431s sys 0m0.218s ``` Could not capture error in api-portal as it started reconcilation automatically in second. total time taken in TAP installation: 5m DNS configuration: 5m (used existing DNS zone and need TAP update with new domain) ## AZURE on separate doc **login to azure and create subscription and resourcegroup:** 1m **cluster creation:** 4.5m ## TKG **Image relocation:** 113m (fails 2 times) ``` export IMGPKG_REGISTRY_HOSTNAME_0=registry.tanzu.vmware.com export IMGPKG_REGISTRY_USERNAME_0=ktanushree@vmware.com export IMGPKG_REGISTRY_PASSWORD_0=Tanu@123 export IMGPKG_REGISTRY_HOSTNAME_1=harbor-repo.vmware.com export IMGPKG_REGISTRY_USERNAME_1=ktanushree export IMGPKG_REGISTRY_PASSWORD_1=Vmw@0222#tanu export INSTALL_REGISTRY_USERNAME=ktanushree export INSTALL_REGISTRY_PASSWORD=Vmw@0222#tanu export INSTALL_REGISTRY_HOSTNAME=harbor-repo.vmware.com export TAP_VERSION=1.5.0 export INSTALL_REPO=carvelind # repo in harbor for which I have admin permission ``` **Pre-install setup:** env, create secret, add repo -> 1.5m **create values.yaml:** 5m **tap install:** 9m (build-service took bit more time to recover after failure) api-portal always failing with below error and after second it start reconciling automatically (without any change) ``` $ kubectl describe pkgi api-portal -n tap-install Name: api-portal Namespace: tap-install Labels: kapp.k14s.io/app=1681889254528350007 kapp.k14s.io/association=v1.31801edfd2d246e80e8529593f369f83 Annotations: kapp.k14s.io/change-rule: delete before deleting serviceaccount kapp.k14s.io/change-rule.cert-manager: upsert after upserting cert-manager kapp.k14s.io/change-rule.telemetry: upsert after upserting telemetry kapp.k14s.io/identity: v1;tap-install/packaging.carvel.dev/PackageInstall/api-portal;packaging.carvel.dev/v1alpha1 kapp.k14s.io/original: {"apiVersion":"packaging.carvel.dev/v1alpha1","kind":"PackageInstall","metadata":{"annotations":{"kapp.k14s.io/change-rule":"delete before... kapp.k14s.io/original-diff-md5: 58e0494c51d30eb3494f7c9198986bb9 API Version: packaging.carvel.dev/v1alpha1 Kind: PackageInstall Metadata: Creation Timestamp: 2023-04-19T07:30:51Z Finalizers: finalizers.packageinstall.packaging.carvel.dev/delete Generation: 1 Managed Fields: API Version: packaging.carvel.dev/v1alpha1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:kapp.k14s.io/change-rule: f:kapp.k14s.io/change-rule.cert-manager: f:kapp.k14s.io/change-rule.telemetry: f:kapp.k14s.io/identity: f:kapp.k14s.io/original: f:kapp.k14s.io/original-diff-md5: f:labels: .: f:kapp.k14s.io/app: f:kapp.k14s.io/association: f:spec: .: f:packageRef: .: f:refName: f:versionSelection: .: f:constraints: f:serviceAccountName: f:values: Manager: kapp Operation: Update Time: 2023-04-19T07:30:51Z API Version: packaging.carvel.dev/v1alpha1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: v:"finalizers.packageinstall.packaging.carvel.dev/delete": Manager: kapp-controller Operation: Update Time: 2023-04-19T07:30:52Z API Version: packaging.carvel.dev/v1alpha1 Fields Type: FieldsV1 fieldsV1: f:status: .: f:conditions: f:friendlyDescription: f:lastAttemptedVersion: f:observedGeneration: f:usefulErrorMessage: f:version: Manager: kapp-controller Operation: Update Subresource: status Time: 2023-04-19T07:30:56Z Resource Version: 311572 UID: 0bec7baa-d8f2-462e-bcda-144564eacd1b Spec: Package Ref: Ref Name: api-portal.tanzu.vmware.com Version Selection: Constraints: 1.3.0 Service Account Name: tap-install-sa Values: Secret Ref: Name: api-portal-values-ver-1 Status: Conditions: Message: Error (see .status.usefulErrorMessage for details) Status: True Type: ReconcileFailed Friendly Description: Reconcile failed: Error (see .status.usefulErrorMessage for details) Last Attempted Version: 1.3.0 Observed Generation: 1 Useful Error Message: kapp: Error: Expected to find kind 'projectcontour.io/v1/HTTPProxy', but did not: - Kubernetes API server did not have matching apiVersion + kind - No matching CRD was found in given configuration Version: 1.3.0 Events: <none> ``` **Build service also failed with error and then start reconciling automatically after few seconds:** ``` Status: Conditions: Message: Error (see .status.usefulErrorMessage for details) Status: True Type: ReconcileFailed Friendly Description: Reconcile failed: Error (see .status.usefulErrorMessage for details) Last Attempted Version: 1.10.8 Observed Generation: 1 Useful Error Message: kapp: Error: waiting on reconcile tanzunetdependencyupdater/dependency-updater (buildservice.tanzu.vmware.com/v1alpha1) namespace: build-service: Finished unsuccessfully (Encountered failure condition Ready == False: CannotImportDescriptor (message: PUT https://harbor-repo.vmware.com/v2/carvelind/build-service/manifests/latest: multiple errors returned: MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:7cc4c1452eb7f0216be80936d9f9255b897332989fecd0489c82bb2f898c5418; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:0057b8952a41a2b717ceb80349588415f9433aa73fd0080911041f4308eb6cc8; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:a775b9fd3da6efc1ca5129ba7791235e2052b990d99e98be23d13da993cd6080; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:e0b9091c372073eac92103f690826fc0c4e8a38c990d1dc3bab51c2f14e4cd99; MANIFEST_BLOB_UNKNOWN: blob unknown to registry; sha256:bdbca6a73be9e8b76c1e565ba846d6a6d69e9b82e2e68f2beaacb3dd5b930b2b)) Version: 1.10.8 ``` **DNS configuration:** 5m ## Script related rough work **list of apps name in TAP:** **Apps Name** -----------------------**Namespace** 1. accelerator -------------------- accelerator-system 1. api-auto-registration ----------- api-auto-registration 1. api-portal ---------------------- api-portal 1. appliveview -------------------- app-live-view 1. appliveview-apiserver appliveview-tokens-system (not sure) 1. appliveview-connector app-live-view-connector 1. appliveview-conventions app-live-view-conventions 1. appsso appsso 1. bitnami-services 1. buildservice build-service 1. cartographer cartographer-system 1. cert-manager cert-manager 1. cnrs 1. contour 3. crossplane 4. developer-conventions developer-conventions 5. eventing 6. fluxcd-source-controller flux-system 7. grype 8. learningcenter 9. learningcenter-workshops 10. metadata-store ------------------ metadata-store 11. namespace-provisioner 12. ootb-delivery-basic 13. ootb-supply-chain-basic 14. ootb-templates 15. policy-controller 16. scanning 17. service-bindings 18. services-toolkit ----------------- services-toolkit 19. source-controller 20. spring-boot-conventions ----------- spring-boot-convention 21. tap ----------------------------- tap-namespace-provisioning 22. tap-auth 23. tap-gui tap-gui 24. tap-telemetry tap-telemetry, vmware-system-telemetry 25. tekton-pipelines tekton-pipelines, tekton-pipelines-resolvers | Op | Condition | Behavior | Affecting annotations | |---|---|---|---| | create | resource present in manifest, but not present in cluster | create resource, see ... | ... | | update | resource exists in cluster but is different than manifest, for differencing see ... | patch resource | ... | | delete | resource not present in manifest, but [labelled as owned](./label-ownership) in cluster | delete resource | ... | | noop | resource in cluster and manifest match, for diff see ... | no change | ... | | exists | resource exists in manifest and cluster, manifest contents ignored | no change | ... | ## TAP 1.5.1 (to verify fixes) ### Azure **image relocation:** ``` export IMGPKG_REGISTRY_HOSTNAME_0=registry.tanzu.vmware.com export IMGPKG_REGISTRY_USERNAME_0=ktanushree@vmware.com export IMGPKG_REGISTRY_PASSWORD_0=Tanu@123 export IMGPKG_REGISTRY_HOSTNAME_1=tapimages151.azurecr.io export IMGPKG_REGISTRY_USERNAME_1=tapimages151 export IMGPKG_REGISTRY_PASSWORD_1=SrV/eZQYM8oSlzjRnthcYDBgTs+BD8MbQxyefaHMnM+ACRCNx0hH export INSTALL_REGISTRY_USERNAME=tapimages151 export INSTALL_REGISTRY_PASSWORD=SrV/eZQYM8oSlzjRnthcYDBgTs+BD8MbQxyefaHMnM+ACRCNx0hH export INSTALL_REGISTRY_HOSTNAME=tapimages151.azurecr.io export TAP_VERSION=1.5.1 export INSTALL_REPO=tap-images ```