# Ruck Rover 2022-09-09 to 2022-09-12 ###### tags: `ruck_rover` ###### Next RR notes: https://hackmd.io/dKeK6zo9R66heikGyCb4NA ###### Previous RR notes: https://hackmd.io/0RguTgCAQsiZz_SnF5209A ##### ruck: Bhagyashris, rover: Jakob [RDO Cockpit](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1) / [RHOS Cockpit](http://tripleo-cockpit.lab4.eng.bos.redhat.com) [RDO Promoter](http://promoter.rdoproject.org/promoter_logs/) / [RHOS Promoter](http://10.0.110.143/promoter_logs/) [OpenStack Program Meeting 2022]( https://docs.google.com/document/d/1n6ArkMh68R9zivjlyGbpedkggk1wMwEIcrMZSN2uIjc/edit) Zuul Status: * [opendev.org:openstack](https://zuul.opendev.org/t/openstack/status/) * [rdoproject.org:rdoproject.org](https://review.rdoproject.org/zuul/status) * [redhat.com:tripleo-ci-internal](https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status) --- ## 2022-09-12 ### Reruns and Investigations: (crossed out := known bugs recorded above) **NOTE:** Watch for running `testproject` jobs on https://review.rdoproject.org/zuul/status and https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status. * [Promotions](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1) - C9 Master + last_promotion: 2022-09-09 + [hash `bb8b8889ee54e27a9b34630694628bf3`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_agg_detail.html?ref_hash=bb8b8889ee54e27a9b34630694628bf3) * ~~`periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master`~~: - ~~[LP#1989197](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master/348bd9a/logs/undercloud/var/log/tempest/stestr_results.html.gz)~~ - ~~[LP#1989197](https://logserver.rdoproject.org/56/44656/16/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master/d891202/logs/undercloud/var/log/tempest/stestr_results.html.gz)~~ * ~~`periodic-tripleo-ci-centos-9-scenario010-kvm-standalone-master`~~ (not in criteria): - [~~LP#2105408?~~ intermittent failure?](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario010-kvm-standalone-master/6e4d4e0/logs/undercloud/var/log/containers/libvirt/qemu/instance-00000001.log.txt.gz) * ~~`periodic-tripleo-ci-centos-9-bm_envC-3ctlr_1comp-featureset001-master`~~ (not in criteria): - [`cannot install the best update candidate for package gnutls-utils-3.7.3-10.el9.x86_64`](https://sf.hosted.upshift.rdu2.redhat.com/logs/42/44742/18/check-rdo/periodic-tripleo-ci-centos-9-bm_envC-3ctlr_1comp-featureset001-master/72e9a0a/job-output.txt) * will be fixed by [chandan's revert](https://review.opendev.org/c/openstack/tripleo-quickstart/+/856582) * `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master`: - [intermittent failure `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/bf5d26f/job-output.txt) - [RETRY_LIMIT in `[configure-mirrors : Update yum/dnf cache]`](https://logserver.rdoproject.org/56/44656/16/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/a7c2d28/job-output.txt) - [RETRY_LIMIT in `[configure-mirrors : Update yum/dnf cache]`](https://logserver.rdoproject.org/56/44656/16/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/fd1567e/job-output.txt) * `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`: - [other tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/7438cdc/logs/undercloud/var/log/tempest/stestr_results.html.gz) - [overcloud deploy failed with `Error: duplicate constraint already exists` in `FATAL | Run init bundle puppet on the host for haproxy`](https://logserver.rdoproject.org/56/44656/16/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/88570ac/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * `periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master`: - [LP#2105408](https://sf.hosted.upshift.rdu2.redhat.com/logs/11/44711/91/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/9f2b792/logs/undercloud/var/log/tempest/stestr_results.html) - [LP#2105408](https://sf.hosted.upshift.rdu2.redhat.com/logs/29/426429/4/check/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/4c73b12/logs/undercloud/var/log/containers/libvirt/qemu/instance-00000001.log) - [LP#2105408](https://sf.hosted.upshift.rdu2.redhat.com/logs/29/426429/4/check/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/75f9ed5/logs/undercloud/var/log/containers/libvirt/qemu/instance-00000001.log) * `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master`: - [intermittent failure `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master/b272efe/job-output.txt) - [RETRY_LIMIT in `[configure-mirrors : Update yum/dnf cache]`](https://logserver.rdoproject.org/56/44656/16/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master/aafecdd/job-output.txt) * [reruns 1](https://review.rdoproject.org/r/c/testproject/+/44656) * [reruns 2](https://code.engineering.redhat.com/gerrit/c/testproject/+/426429) - C9 Wallaby + NOTE from #rhos-ops: > rlandy: amoralej|off: dpawlik|off: hmm ... https://trunk.rdoproject.org/centos9-wallaby/component/network/component-ci-testing/commit.yaml has the wrong commit branch > commit_branch: master > should be stable/wallaby - should be safe to skip and promote here - see last few run with different tempest failures) + last_promotion: 2022-09-09 + [hash `f6749f9cda54d017021ab56bf4ec0958`](https://trunk.rdoproject.org/api-centos9-wallaby/api/civotes_agg_detail.html?ref_hash=f6749f9cda54d017021ab56bf4ec0958) * ~~`periodic-tripleo-ci-centos-9-skipped-scenario012-standalone-wallaby`~~ (not in criteria): - [tempest failures](https://logserver.rdoproject.org/openstack-periodic-weekend/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-skipped-scenario012-standalone-wallaby/a1b2c7f/logs/undercloud/var/log/tempest/stestr_results.html.gz) * `periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-wallaby`: - [overcloud deploy failed with `keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate.` in `Clean up legacy Cinder keystone catalog entries`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-wallaby/5ac716b/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [node provisioning failed](https://logserver.rdoproject.org/61/44661/12/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-wallaby/d624dc4/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz) * `periodic-tripleo-ci-centos-9-8-multinode-mixed-os`: - [other tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-8-multinode-mixed-os/4bcfcdc/logs/undercloud/var/log/tempest/stestr_results.html.gz) - [`[undercloud]: FAILED...Invalid response received from the delorean server`](https://logserver.rdoproject.org/61/44661/12/check/periodic-tripleo-ci-centos-9-8-multinode-mixed-os/9212099/job-output.txt) * `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby`: - [intermittent failure `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/7f85478/job-output.txt) - [overcloud deploy failed in `FATAL | Ensure certificate requests` with no error message, wtf?](https://logserver.rdoproject.org/61/44661/12/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/5c566e1/logs/undercloud/home/zuul/overcloud1_deploy.log.txt.gz) * ~~`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby`~~: - ~~[LP#1987092](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/b5abcac/logs/overcloud-controller-1/var/log/containers/stdouts/galera-bundle.log.txt.gz)~~ * `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`: - [intermittent failure `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/7834334/job-output.txt) - [overcloud deploy failed with unknown error](https://logserver.rdoproject.org/61/44661/12/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/1e2e94e/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * ~~`periodic-tripleo-ci-centos-8-9-multinode-mixed-os`~~ (not in criteria): - [otherĀ² tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-stable1-cs8/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-9-multinode-mixed-os/5f64adf/logs/undercloud/var/log/tempest/stestr_results.html) * ~~`periodic-tripleo-ci-centos-9-skipped-standalone-wallaby`~~ (not in criteria): - [otherĀ³ tempest failures](https://logserver.rdoproject.org/openstack-periodic-weekend/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-skipped-standalone-wallaby/e6088b4/logs/undercloud/var/log/tempest/stestr_results.html.gz) * ~~`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby`~~ (not in criteria): - [otherĀ¼ tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby/8037be8/logs/undercloud/var/log/tempest/stestr_results.html.gz) * [reruns 1](https://review.rdoproject.org/r/c/testproject/+/44661) * [reruns 2](https://review.rdoproject.org/r/c/testproject/+/44899) * [reruns 3](https://review.rdoproject.org/r/c/testproject/+/44900) * [reruns 4](https://review.rdoproject.org/r/c/testproject/+/44901) * [reruns 5](https://review.rdoproject.org/r/c/testproject/+/44902) * [reruns 6](https://review.rdoproject.org/r/c/testproject/+/44903) * [reruns 7](https://review.rdoproject.org/r/c/testproject/+/44904) * [reruns 8](https://review.rdoproject.org/r/c/testproject/+/44905) * [reruns 9](https://review.rdoproject.org/r/c/testproject/+/44906) * [reruns 10](https://review.rdoproject.org/r/c/testproject/+/44907) * [reruns 11](https://review.rdoproject.org/r/c/testproject/+/44908) * [reruns 12](https://review.rdoproject.org/r/c/testproject/+/44909) * TAKE THAT, WALLABY!!! * NOTE: f6749f9cda54d017021ab56bf4ec0958 is missing on fs064 - can't get that to pass trying for hash ea8639e875ec8d7ee7158bce2cd3fa2b now https://review.rdoproject.org/r/c/testproject/+/31165 and https://review.rdoproject.org/r/c/testproject/+/36255 - C8 Wallaby + last_promotion: 2022-09-11 + [hash `21d2e33704a2f14d611cf4edff39f70e`](https://trunk.rdoproject.org/api-centos8-wallaby/api/civotes_agg_detail.html?ref_hash=21d2e33704a2f14d611cf4edff39f70e) * ~~`periodic-tripleo-ci-centos-8-skipped-standalone-wallaby`~~ (not in criteria): - [`No container for this DLRN hash`](https://logserver.rdoproject.org/openstack-periodic-weekend/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-skipped-standalone-wallaby/a3ac814/logs/_No_container_for_this_DLRN_hash.log) - C8 Train + last_promotion: 2022-09-09 + [hash `489d88a4b22eb070acc39b218844ac82`](https://trunk.rdoproject.org/api-centos8-train/api/civotes_agg_detail.html?ref_hash=489d88a4b22eb070acc39b218844ac82) * ~~`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-internal-train`~~ (not in criteria): - [tempest failures](https://sf.hosted.upshift.rdu2.redhat.com/logs/13/44713/10/check-rdo/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-internal-train/45f6b89/logs/undercloud/var/log/tempest/stestr_results.html.gz) * `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-train`: - [intermittent (?) failure in `TASK [freeipa-setup : Deploy FreeIPA]`](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-train/1d3008b/job-output.txt) - [overcloud deploy failed with `WSREP replication failed. Check your wsrep connection state and retry the query`](https://logserver.rdoproject.org/90/44690/4/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train/2798945/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * ~~`periodic-tripleo-ci-centos-8-skipped-scenario012-standalone-train`~~ (not in criteria): - [tempest rampage](https://logserver.rdoproject.org/openstack-periodic-weekend/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-skipped-scenario012-standalone-train/fa9b5d2/logs/undercloud/var/log/tempest/stestr_results.html.gz) * ~~`periodic-tripleo-ci-centos-8-skipped-standalone-train`~~ (not in criteria): - [tempest destructiod](https://logserver.rdoproject.org/openstack-periodic-weekend/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-skipped-standalone-train/4e92533/logs/undercloud/var/log/tempest/stestr_results.html.gz) * ~~`periodic-tripleo-ci-centos-8-standalone-full-tempest-api-train`~~: - ~~[tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-api-train/3ebdd17/logs/undercloud/var/log/tempest/stestr_results.html)~~ * [reruns](https://review.rdoproject.org/r/c/testproject/+/44662) - RHOS-17.1 on RHEL-9 **Refer to Bhagyashris' notes below** - RHOS-17.1 on RHEL-8 **Refer to Bhagyashris' notes below** - RHOS-17.0 on RHEL-9 **Refer to Bhagyashris' notes below** - RHOS-16.2 on RHEL-8 **Refer to Bhagyashris' notes below** * [C9 Master](http://dashboard-ci.tripleo.org/d/2tivP9BWz/component-pipeline?orgId=1): - clients + last_promotion: 2022-09-09 + ~~`periodic-tripleo-ci-centos-9-undercloud-upgrade-clients-master`~~ (not in criteria): * [undercloud install failed in `TASK [undercloud-setup : Run the package installation script]` with `Problem: package openstack-heat-api-1:***.el9.noarch requires openstack-heat-common = 1:***.el9, but none of the providers can be installed`](https://logserver.rdoproject.org/openstack-component-clients/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-upgrade-clients-master/cd352ac/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz) - common + last_promotion: 2022-09-11 + ~~`periodic-tripleo-ci-centos-9-undercloud-upgrade-common-master`~~ (not in criteria): * [undercloud install failed in `TASK [undercloud-setup : Run the package installation script]` with `Problem: package openstack-heat-api-1:***.el9.noarch requires openstack-heat-common = 1:***.el9, but none of the providers can be installed`](https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-upgrade-common-master/4ef4df0/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz) - compute + last_promotion: 2022-09-07 + `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master`: * [LP#1989197](https://logserver.rdoproject.org/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master/2fd6041/logs/undercloud/var/log/tempest/stestr_results.html.gz) * [LP#1989197](https://logserver.rdoproject.org/57/44657/11/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master/717f610/logs/undercloud/var/log/tempest/stestr_results.html.gz) - manila + last_promotion: 2022-09-09 + ~~`periodic-tripleo-ci-centos-9-scenario004-standalone-manila-master`~~: * ~~[tempest failures](https://logserver.rdoproject.org/openstack-component-manila/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario004-standalone-manila-master/cd4d74b/logs/undercloud/var/log/tempest/stestr_results.html.gz)~~ - network + last_promotion: 2022-09-08 + ~~`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-master`~~: * ~~[overcloud deploy failed with `Error: duplicate constraint already exists` in `FATAL | Run init bundle puppet on the host for haproxy`](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-master/5672241/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)~~ + ~~`periodic-tripleo-ci-centos-9-undercloud-upgrade-network-master`~~ (not in criteria): * [undercloud install failed in `TASK [undercloud-setup : Run the package installation script]` with `Problem: package openstack-heat-api-1:***.el9.noarch requires openstack-heat-common = 1:***.el9, but none of the providers can be installed`](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-upgrade-network-master/486a62c/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz) + `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master`: * [LP#1989197](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master/dc540aa/logs/undercloud/var/log/tempest/stestr_results.html.gz) * [LP#1989197](https://logserver.rdoproject.org/57/44657/11/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master/b54f490/logs/undercloud/var/log/tempest/stestr_results.html.gz) - octavia + last_promotion: 2022-09-06 + ~~`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-octavia-master`~~ (not in criteria): * [LP#2105408](https://sf.hosted.upshift.rdu2.redhat.com/logs/11/44711/71/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-octavia-master/bbc6a88/logs/undercloud/var/log/containers/libvirt/qemu/instance-00000001.log) - tripleo + last_promotion: 2022-09-08 + ~~`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-master`~~: * ~~[overcloud deploy failed with `Error: .../Tripleo::Pacemaker::Haproxy_with_vip.../ensure: change from 'absent' to 'present' failed` in `FATAL | Run init bundle puppet on the host for haproxy`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-master/818fcb4/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)~~ - [reruns](https://review.rdoproject.org/r/c/testproject/+/44657) * [C9 Wallaby](http://dashboard-ci.tripleo.org/d/UcMR8py7z/component-pipeline-wallaby-centos9?orgId=1): - common + last_promotion: 2022-09-10 + ~~`periodic-tripleo-ci-centos-9-8-multinode-mixed-os-common-wallaby`~~ (not in criteria): * [intermittent (?) failure in `TASK [install and run os-net-config]`: `ModuleNotFoundError: No module named 'oslo_concurrency'`](https://logserver.rdoproject.org/84/44684/7/check/periodic-tripleo-ci-centos-9-8-multinode-mixed-os-common-wallaby/5fa2da2/job-output.txt) - network + last_promotion: 2022-09-03 + [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby): * older runs: - 2022-09-09: [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/58/44658/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby/ea08fd8/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - 2022-09-09: [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby/31f80d8/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - 2022-09-10: [overcloud deploy failed with `ERROR: Can't run container rabbitmq_bootstrap`](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby/0b473bf/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * [overcloud deploy failed with `Error: error creating container storage: the container name "container-puppet-swift" is already in use by ***. You have to remove that container to be able to reuse that name: that name is already in use`](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby/7ed1819/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * [tempest failures](https://logserver.rdoproject.org/58/44658/6/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby/8ada848/logs/undercloud/var/log/tempest/stestr_results.html.gz) - tripleo + last_promotion: 2022-09-06 + ~~`periodic-tripleo-ci-centos-9-8-multinode-mixed-os-tripleo-wallaby`~~ (not in criteria): * [intermittent (?) FAILURE in `TASK [install and run os-net-config]`: `nothing provides python3.9dist(pbr) >= 2 needed by os-net-config-***.el9.noarch`](https://logserver.rdoproject.org/84/44684/7/check/periodic-tripleo-ci-centos-9-8-multinode-mixed-os-tripleo-wallaby/4383f6f/job-output.txt) + ~~`periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-tripleo-wallaby`~~: * ~~[intermittent (?) FAILURE in `TASK [Install virtualenv]`: `Failed to download packages: python3-wheel-wheel-***.el9.noarch`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-tripleo-wallaby/aba5598/job-output.txt)~~ + ~~`periodic-tripleo-ci-centos-9-containers-multinode-tripleo-wallaby`~~: * ~~[intermittent (?) FAILURE in `TASK [repo-setup : Create repo setup script]`: `Failed to connect to the host via ssh: kex_exchange_identification: read: Connection reset by peer`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-containers-multinode-tripleo-wallaby/9e2d128/job-output.txt)~~ + ~~`periodic-tripleo-ci-centos-8-9-multinode-mixed-os-tripleo-wallaby`~~ (not in criteria): * [intermittent (?) FAILURE in `TASK [install and run os-net-config]`: `nothing provides python3.9dist(pbr) >= 2 needed by os-net-config-***.el9.noarch`](https://logserver.rdoproject.org/84/44684/7/check/periodic-tripleo-ci-centos-8-9-multinode-mixed-os-tripleo-wallaby/cc1f140/job-output.txt) + ~~`periodic-tripleo-ci-centos-9-containers-multinode-tripleo-wallaby-validation`~~: * ~~[overcloud deploy failed in `FATAL | Wait for puppet host configuration to finish` with `puppet-user: Error: Execution of '/bin/dnf -d 0 -e 1 -y install net-snmp' returned 1: Error: Error downloading packages:`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-containers-multinode-tripleo-wallaby-validation/7018188/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)~~ - [reruns](https://review.rdoproject.org/r/c/testproject/+/44658) * [C8 Wallaby](http://dashboard-ci.tripleo.org/d/ZLvFbT9Mz/component-pipeline-wallaby?orgId=1): - all **GREEN** šŸ˜± - ~~[reruns](https://review.rdoproject.org/r/c/testproject/+/44659)~~ * [C8 Train](http://dashboard-ci.tripleo.org/d/aJeQpzVGz/component-pipeline-train?orgId=1): - network + last_promotion: 2022-09-09 + ~~`periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train`~~ (not in criteria): * [tempest failures](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train/5cd5938/logs/undercloud/var/log/tempest/stestr_results.html) - ~~[reruns](https://review.rdoproject.org/r/c/testproject/+/44690)~~ * [RHOS-17.0 on RHEL-9](http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/lF7RUpsnk/rhel9-rhos17-full-component-pipeline?orgId=1): * **Refer to Bhagyashris' notes below** * [RHOS-16.2 on RHEL-8](http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/KyHCwLHMk/rhos-16-2-full-component-pipeline?orgId=1): * **Refer to Bhagyashris' notes below** ### Downstream: * rhos16.2 on rhel8: * Promotion: 12-sept-2022 * blocked due to sc010 re-running jobs here: https://code.engineering.redhat.com/gerrit/c/testproject/+/422400/ * rhos17.1 on rhel8 * Promotion: 12-sept-2022 * rhos17 on rhel9: * Promotion: 11-Sept-2022 * failed fs001 job so re-ran here : https://code.engineering.redhat.com/gerrit/c/testproject/+/422400/10#message-60300b428a51d8e0c5639502640ead15c847b402 : **PASSED** * rhos17.1 on rhel9: * Promoted: 12-sept-2022 * blocked due full tempest api so re-ran here: https://code.engineering.redhat.com/gerrit/c/testproject/+/422400/10#message-60300b428a51d8e0c5639502640ead15c847b402 : **PASSED** * jenkins job not triggerd: * Note : Pinged atila: - After discussion with atila we got to know that currently we don't have trigger jobs for 17.1 on rhel9 and 8 so i am adding trigger job for 17.1 on 9 and 8 here: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/428003 * ~~fix for trigger jekins component rhso17.1 on rhel9 jobs is here: https://code.engineering.redhat.com/gerrit/c/rhos-qe-jenkins/+/428056 ~~ ### Downstream component: ### Known Bugs * https://bugs.launchpad.net/tripleo/+bug/1960355 - > Tempest tests test_oauth1_tokens.OAUTH1TokensTest are failing on tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-(master|wallaby) - affecting OSP17.1 as well * https://bugs.launchpad.net/tripleo/+bug/1987092 :warning: **BLOCKER** :warning: - Pacemaker performance causes intermittent galera issues in loaded CI env - jobs fail on `Check Keystone public endpoint status` or in other places - [zuul/builds](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-validation-master&skip=0) - [affecting non-validation jobs such as periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby](https://bugs.launchpad.net/tripleo/+bug/1987092/comments/18) * https://bugzilla.redhat.com/2105408 :warning: **BLOCKER** :warning: - > RHOSP16.2 - Octavia tempest test "test_load_balancer_ipv4_CRUD" fails due to a kernel stacktrace (in a nested KVM setup) - > reproducible only on RHEL 8.2 and RHEL 8.4 ... RHEL 8.6 is unaffected. So, please update the affected CI systems to RHEL 8.6. - [latest status is tracked in cix card](https://trello.com/c/msIodUbD/2616-cixbz2105408osp162rhel8octavia-tempest-test-loadbalancerscenariotesttestloadbalanceripv4crud-failing-on-load-balancer-creation-t) - blocking integration lines of c9 master (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master`) and c9 wallaby (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-wallaby`) - [workaround (specifying `cpu_models` in libvirt)]( https://code.engineering.redhat.com/gerrit/c/openstack-tripleo-heat-templates/+/425806/) is [currently being tested](https://code.engineering.redhat.com/gerrit/c/testproject/+/425432/) * https://bugs.launchpad.net/tripleo/+bug/1988514 - [Trello](https://trello.com/c/TAL1jQcB/2692-cixlp1988514tripleociproa-source-image-rejected-none-of-the-signatures-were-accepted-reasons-open-etc-pki-rpm-gpg-rpm-gpg-key-re) - On CIX board in lane `Waiting on external dependencies` * https://bugs.launchpad.net/tripleo/+bug/1987323 - > FIPS + Manila + Ceph Quincy - > ... known issue on the ceph side, ... already merged in master/quincy ... > 17.2.4 should solve this issue, so we'll check during the next promotion. - On CIX board in lane `Waiting on external dependencies` - Not a blocker * https://bugs.launchpad.net/tripleo/+bug/1988810 - Revert https://review.rdoproject.org/r/c/rdo-jobs/+/44788 * https://bugs.launchpad.net/tripleo/+bug/1989197 - > Tempest test `neutron_tempest_plugin.api.test_port_forwardings.PortForwardingTestJSON` fails on `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master` * https://bugs.launchpad.net/tripleo/+bug/1989341 - > Tempest tests "tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON" etc. fail on {periodic-,}tripleo-ci-centos-8-9-multinode-mixed-os due to ovn controller/north version mismatch * https://bugs.launchpad.net/tripleo/+bug/1989247 - > overcloud deploy fails with `sudo: unable to send audit message: Operation not permitted` - > `Create containers managed by Podman for ... Failed containers: rabbitmq_bootstrap` (or others) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`](https://logserver.rdoproject.org/56/44656/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/007c1da/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master`](https://logserver.rdoproject.org/57/44657/9/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master/3fb6e42/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) ### Intermittent Failures * https://bugs.launchpad.net/tripleo/+bug/1987616 - > cs9 ovb fs01 clients wallaby is failing with error - 'neutron.agent.dhcp.agent oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply' - no CIX card * https://bugs.launchpad.net/tripleo/+bug/1938987 - periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train - > ERROR keystone.server.flask.request_processing.middleware.auth_context oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (1047, 'WSREP has not yet prepared node for application use') - [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train`](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/bd0a94d/logs/overcloud-controller-2/var/log/containers/keystone/keystone.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train`](https://logserver.rdoproject.org/62/44662/6/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/5753657/logs/overcloud-controller-0/var/log/containers/keystone/keystone.log.txt.gz) * `TASK [freeipa-setup : Run the tripleo-ipa preparation script]` - [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/205c81b/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/f940a55/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby/bfa6f03/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/40/44840/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/57331f2/job-output.txt) * `Connection failure: The read operation timed out` in `TASK [Get current autoholds]` - [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/c3a98c9/job-output.txt) * `Create containers managed by Podman for ... Failed containers: rabbitmq_bootstrap` (or others) - > this has different root causes, watch out for log details! - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/4926640/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [FAILURE in `tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/df357c2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * `FATAL | Wait for puppet host configuration to finish | undercloud | error`... - `TASK [tripleo.operator.tripleo_undercloud_install : undercloud install]` in [job-output.txt](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/job-output.txt) - `puppet-user: Error: Execution of '/bin/dnf -d 0 -e 1 -y install net-snmp' returned 1: Error: Error downloading packages:` in [logs/undercloud/home/zuul/undercloud_install.log.txt.gz](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz) - [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz) * `Exception: Command '['ipmitool', ..., 'status']' timed out after 60 seconds"` during overcloud deploy - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/0a8588f/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz) ### Fixed * https://bugs.launchpad.net/tripleo/+bug/1988738 * Octavia ovn driver function create_vip_port wasn't updated with latest octavia changes * [job passes since 2022-09-08](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario010-ovn-provider-standalone-master) --- ## 2022-09-09 ### Reruns and Investigations: (crossed out := known bugs recorded above) **NOTE:** Watch for running `testproject` jobs on https://review.rdoproject.org/zuul/status and https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status. * [Promotions](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1) - C9 Master: + last_promotion: 2022-08-31 + [hash 3edefe9728ecab11e979d22a0d03263d](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_agg_detail.html?ref_hash=3edefe9728ecab11e979d22a0d03263d) * `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master`: - [skiplist patch](https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/856764) - [reruns with Depends-On skiplist patch](https://review.rdoproject.org/r/c/testproject/+/44656) * `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`: - [patch to remove c9 master fs35 from promotion criteria](https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/44863) * [reruns 1](https://review.rdoproject.org/r/c/testproject/+/44656) * [reruns 2](https://code.engineering.redhat.com/gerrit/c/testproject/+/426429) + [hash bb23ee0d771f83919a7fc55f41bfe6ad](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_agg_detail.html?ref_hash=bb23ee0d771f83919a7fc55f41bfe6ad) * **WE DO NOT TRY TO PROMOTE THIS HASH!** * **SEE PREVIOUS HASH ABOVE FOR TODAYS PROMOTION TARGET** * `periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master`: - ~~[overcloud deploy failed with `Error: 'haproxy-bundle' already exists` in `FATAL | Run init bundle puppet on the host for haproxy`](https://logserver.rdoproject.org/29/44829/4/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master/fb38db8/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)~~ - [overcloud deploy failed with `Create containers managed by Podman for ... Failed containers`](https://logserver.rdoproject.org/56/44656/13/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master/948724b/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * ~~`periodic-tripleo-ci-centos-9-bm_envC-3ctlr_1comp-featureset001-master`~~ (not in criteria): - [failed in `TASK [repo-setup : Setup repos on live host]` with `package gnutls-utils-3.7.6-7.el9.x86_64 requires gnutls(x86-64) = 3.7.6-7.el9, but none of the providers can be installed`](https://sf.hosted.upshift.rdu2.redhat.com/logs/42/44742/12/check-rdo/periodic-tripleo-ci-centos-9-bm_envC-3ctlr_1comp-featureset001-master/0b350b7/job-output.txt) - will be fixed by [chandan's revert](https://review.opendev.org/c/openstack/tripleo-quickstart/+/856582) * `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master`: - ~~[tempest failure](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master/2d35775/logs/undercloud/var/log/tempest/stestr_results.html.gz)~~ - [overcloud deploy failed with `Create containers managed by Podman for ... Failed containers`](https://logserver.rdoproject.org/56/44656/13/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master/1fe781e/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master`: - [LP#1989197](https://logserver.rdoproject.org/29/44829/4/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master/8101527/logs/undercloud/var/log/tempest/stestr_results.html.gz) - [LP#1989197](https://logserver.rdoproject.org/56/44656/13/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master/727e0e0/logs/undercloud/var/log/tempest/stestr_results.html.gz) * ~~`periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master`~~: - ~~[otherĀ² tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master/1021480/logs/undercloud/var/log/tempest/stestr_results.html.gz)~~ * ~~`periodic-tripleo-ci-centos-9-scenario010-kvm-standalone-master`~~ (not in criteria): - ~~[otherĀ³ tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario010-kvm-standalone-master/95f29a7/logs/undercloud/var/log/tempest/stestr_results.html.gz)~~ * `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`: - ~~[overcloud deploy failed with `ERROR: Can't run container mysql_wait_bundle`](https://logserver.rdoproject.org/29/44829/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/ef5049f/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)~~ - [otherĀ¼ tempest failures](https://logserver.rdoproject.org/56/44656/13/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/82fb95d/logs/undercloud/var/log/tempest/stestr_results.html.gz) * `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master`: - ~~[undercloud install failed in `FATAL | ensure apache is installed` with `Failed to download packages: apr-util-1.6.1-20.el9.x86_64`](https://logserver.rdoproject.org/29/44829/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/c4bca75/logs/undercloud/home/zuul/undercloud_install.log.txt.gz)~~ - [tempest failures](https://logserver.rdoproject.org/56/44656/13/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/84ae0b6/logs/undercloud/var/log/tempest/stestr_results.html.gz) - C9 Wallaby + last_promotion: 2022-09-05 + [hash](https://trunk.rdoproject.org/api-centos9-wallaby/api/civotes_agg_detail.html?ref_hash=a29104cf0a2a21bb5b863335b204ba1f) + `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`: * [intermittent FAILURE in `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`](https://logserver.rdoproject.org/40/44840/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/57331f2/job-output.txt) * [intermittent FAILURE in `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`](https://logserver.rdoproject.org/61/44661/9/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/d269f17/job-output.txt) + `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby`: * [overcloud deploy failed with `Error: duplicate constraint already exists` in `FATAL | Run init bundle puppet on the host for haproxy`](https://logserver.rdoproject.org/40/44840/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/05be30f/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * [overcloud deploy failed with `Error: .../Tripleo::Pacemaker::Haproxy_with_vip.../ensure: change from 'absent' to 'present' failed` in `FATAL | Run init bundle puppet on the host for haproxy`](https://logserver.rdoproject.org/61/44661/9/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/18312d6/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) + ~~`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby`~~ (not in criteria): * [intermittent FAILURE during overcloud deploy: `Create containers managed by Podman for ... Failed containers:`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby/e510014/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) + ~~`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby`~~: * ~~[intermittent FAILURE during overcloud node provisioning ](https://logserver.rdoproject.org/40/44840/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/7c9467e/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz)~~ + [reruns](https://review.rdoproject.org/r/c/testproject/+/44661) - C8 Wallaby + last_promotion: 2022-09-07 + [hash](https://trunk.rdoproject.org/api-centos8-wallaby/api/civotes_agg_detail.html?ref_hash=b600fc85484a3e36e58d50642bc727b2) - C8 Train + last_promotion: 2022-09-07 + [hash](https://trunk.rdoproject.org/api-centos8-train/api/civotes_agg_detail.html?ref_hash=4373585269a00ee0bafb01a7e4bf205b) + `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train`: * [tempest failures](https://logserver.rdoproject.org/17/38017/28/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/64be90e/logs/undercloud/var/log/tempest/stestr_results.html.gz) + [reruns](https://review.rdoproject.org/r/c/testproject/+/44662) - RHOS-17.1 on RHEL-9 **Refer to Bhagyashris' notes below** - RHOS-17.1 on RHEL-8 **Refer to Bhagyashris' notes below** - RHOS-17.0 on RHEL-9 **Refer to Bhagyashris' notes below** - RHOS-16.2 on RHEL-8 **Refer to Bhagyashris' notes below** * [C9 Master](http://dashboard-ci.tripleo.org/d/2tivP9BWz/component-pipeline?orgId=1): - clients + last_promotion: 2022-09-09 + `periodic-tripleo-ci-centos-9-undercloud-upgrade-clients-master`: * [undercloud install failed in `TASK [undercloud-setup : Run the package installation script]` with `Problem: package openstack-heat-monolith-1:16.0.1-0.20220817062359.d0e2ded.el9.noarch requires openstack-heat-common = 1:16.0.1-0.20220817062359.d0e2ded.el9, but none of the providers can be installed`](https://logserver.rdoproject.org/openstack-component-clients/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-upgrade-clients-master/83c952c/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz) - common + last_promotion: 2022-09-09 + `periodic-tripleo-ci-centos-9-undercloud-upgrade-common-master`: * [undercloud install failed in `TASK [undercloud-setup : Run the package installation script]` with `Problem: package openstack-heat-api-1:16.0.1-0.20220817062359.d0e2ded.el9.noarch requires openstack-heat-common = 1:16.0.1-0.20220817062359.d0e2ded.el9, but none of the providers can be installed`](https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-upgrade-common-master/b5cc2f0/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz) - compute + last_promotion: 2022-09-07 + `periodic-tripleo-ci-centos-9-scenario012-standalone-compute-master`: * [tempest failures](https://logserver.rdoproject.org/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario012-standalone-compute-master/3c8a087/logs/undercloud/var/log/tempest/stestr_results.html.gz) + `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master`: * [other tempest failures](https://logserver.rdoproject.org/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master/c1ad9b9/logs/undercloud/var/log/tempest/stestr_results.html.gz) - network + last_promotion: 2022-09-08 + `periodic-tripleo-ci-centos-9-standalone-network-master`: * [otherĀ² tempest failures](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-network-master/b0bedab/logs/undercloud/var/log/tempest/stestr_results.html.gz) + `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-master`: * [otherĀ³ tempest failures](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-master/6e5bc31/logs/undercloud/var/log/tempest/stestr_results.html.gz) + `periodic-tripleo-ci-centos-9-containers-multinode-network-master`: * [overcloud deploy failed in `FATAL | Check Keystone internal endpoint status`](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-containers-multinode-network-master/7c44280/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-undercloud-upgrade-network-master`: * [undercloud install failed in `TASK [undercloud-setup : Run the package installation script]` with `Problem: package openstack-heat-api-1:16.0.1-0.20220817062359.d0e2ded.el9.noarch requires openstack-heat-common = 1:16.0.1-0.20220817062359.d0e2ded.el9, but none of the providers can be installed`](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-upgrade-network-master/195315b/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz) + `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master`: * [otherĀ½ tempest failures](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master/59ef943/logs/undercloud/var/log/tempest/stestr_results.html.gz) - octavia + last_promotion: 2022-09-06 + `periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-octavia-master`: * [BZ#2105408](https://sf.hosted.upshift.rdu2.redhat.com/logs/11/44711/44/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-octavia-master/09bcb5e/logs/undercloud/var/log/containers/libvirt/qemu/instance-00000001.log) - security + last_promotion: 2022-09-05 + `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master`: * [intermittent FAILURE: 'arecord' not found.](https://logserver.rdoproject.org/openstack-component-security/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master/6146852/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - tripleo + last_promotion: 2022-09-08 + `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-master`: * [tempest failures](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-master/05c8d67/logs/undercloud/var/log/tempest/stestr_results.html.gz) + `periodic-tripleo-ci-centos-9-standalone-tripleo-master`: * [undercloud deploy failed with `FATAL | Check Keystone public endpoint status`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-tripleo-master/611652a/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) - [reruns](https://review.rdoproject.org/r/c/testproject/+/44657) * [C9 Wallaby](http://dashboard-ci.tripleo.org/d/UcMR8py7z/component-pipeline-wallaby-centos9?orgId=1): - baremetal + last_promotion: 2022-09-01 + `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-baremetal/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/e89614e/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - common + last_promotion: 2022-09-08 + `periodic-tripleo-ci-centos-9-scenario002-standalone-common-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario002-standalone-common-wallaby/8699ae8/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-containers-multinode-common-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-containers-multinode-common-wallaby/64ebdee/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-standalone-common-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-common-wallaby/2215d5f/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-scenario001-standalone-common-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario001-standalone-common-wallaby/464029e/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-scenario004-standalone-common-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario004-standalone-common-wallaby/9f4230a/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-scenario003-standalone-common-wallaby`: * [standalone deploy failed with `Error running container image prepare: Modifying image ... failed`](https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario003-standalone-common-wallaby/c31cd34/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) - compute + last_promotion: 2022-08-29 + `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-compute-wallaby`: * [overcloud node provisioning failed](https://logserver.rdoproject.org/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-compute-wallaby/5ac6e30/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz) + `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-wallaby`: * [overcloud deploy failed with `Error: OCI runtime error: crun: fork: Cannot allocate memory`](https://logserver.rdoproject.org/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-wallaby/60e025b/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-standalone-full-tempest-scenario-compute-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-scenario-compute-wallaby/72454a5/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) - network + last_promotion: 2022-09-03 + `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby`: * [LP#1987092](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby/ab31a2d/logs/overcloud-controller-0/var/log/containers/stdouts/galera-bundle.log.txt.gz) - tripleo + last_promotion: 2022-09-06 + `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/0741054/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-containers-multinode-tripleo-wallaby-validation`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-containers-multinode-tripleo-wallaby-validation/1375c7b/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-tripleo-wallaby`: * [intermittent FAILURE: 'arecord' not found.](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-tripleo-wallaby/fe01aa2/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-scenario004-standalone-tripleo-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario004-standalone-tripleo-wallaby/5594331/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-containers-multinode-tripleo-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-containers-multinode-tripleo-wallaby/c3b6f8f/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-standalone-tripleo-wallaby`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-tripleo-wallaby/422d974/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) + `periodic-tripleo-ci-centos-9-standalone-tripleo-wallaby-validation`: * [overcloud deploy failed with `Error: can only create exec sessions on running containers: container state improper`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-tripleo-wallaby-validation/07aac25/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) - [reruns](https://review.rdoproject.org/r/c/testproject/+/44658) * [C8 Wallaby](http://dashboard-ci.tripleo.org/d/ZLvFbT9Mz/component-pipeline-wallaby?orgId=1): - network + last_promotion: 2022-09-06 + `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-wallaby`: * [overcloud node provisioning failed](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-wallaby/c4e99e6/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz) - tripleo + last_promotion: 2022-09-08 + `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`: * [`Overcloud stack: FAILED. MAC conflict between hosts.`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/463af23/logs/_Overcloud_stack__FAILED__MAC_conflict_between_hosts.log) - [reruns](https://review.rdoproject.org/r/c/testproject/+/44659) * [C8 Train](http://dashboard-ci.tripleo.org/d/aJeQpzVGz/component-pipeline-train?orgId=1): - compute + last_promotion: 2022-08-23 + `periodic-tripleo-ci-centos-8-standalone-full-tempest-api-compute-train`: * [tempest failures](https://logserver.rdoproject.org/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-api-compute-train/c182da8/logs/undercloud/var/log/tempest/stestr_results.html) - network + last_promotion: 2022-09-06 + `periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train`: * [similar tempest failures to compute job](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train/9596aaf/logs/undercloud/var/log/tempest/stestr_results.html) - tripleo + last_promotion: 2022-09-06 + `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train`: * [overcloud deploy failed with `FATAL | Run NetworkConfig script | ... "async task did not complete within the requested time`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train/68350a1/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [reruns](https://review.rdoproject.org/r/c/testproject/+/44690) * [RHOS-17.0 on RHEL-9](http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/lF7RUpsnk/rhel9-rhos17-full-component-pipeline?orgId=1): * **Refer to Bhagyashris' notes below** * [RHOS-16.2 on RHEL-8](http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/KyHCwLHMk/rhos-16-2-full-component-pipeline?orgId=1): * **Refer to Bhagyashris' notes below** ### Known Bugs * https://bugs.launchpad.net/tripleo/+bug/1988738 * Octavia ovn driver function create_vip_port wasn't updated with latest octavia changes - Waiting on master promotion * https://bugs.launchpad.net/tripleo/+bug/1960355 - > Tempest tests test_oauth1_tokens.OAUTH1TokensTest are failing on tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-(master|wallaby) - affecting OSP17.1 as well * https://bugs.launchpad.net/tripleo/+bug/1987092 :warning: **BLOCKER** :warning: - Pacemaker performance causes intermittent galera issues in loaded CI env - jobs fail on `Check Keystone public endpoint status` or in other places - [zuul/builds](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-validation-master&skip=0) - [affecting non-validation jobs such as periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby](https://bugs.launchpad.net/tripleo/+bug/1987092/comments/18) * https://bugzilla.redhat.com/2105408 :warning: **BLOCKER** :warning: - > RHOSP16.2 - Octavia tempest test "test_load_balancer_ipv4_CRUD" fails due to a kernel stacktrace (in a nested KVM setup) - > reproducible only on RHEL 8.2 and RHEL 8.4 ... RHEL 8.6 is unaffected. So, please update the affected CI systems to RHEL 8.6. - [latest status is tracked in cix card](https://trello.com/c/msIodUbD/2616-cixbz2105408osp162rhel8octavia-tempest-test-loadbalancerscenariotesttestloadbalanceripv4crud-failing-on-load-balancer-creation-t) - blocking integration lines of c9 master (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master`) and c9 wallaby (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-wallaby`) - [workaround (specifying `cpu_models` in libvirt)]( https://code.engineering.redhat.com/gerrit/c/openstack-tripleo-heat-templates/+/425806/) is [currently being tested](https://code.engineering.redhat.com/gerrit/c/testproject/+/425432/) * https://bugs.launchpad.net/tripleo/+bug/1988514 - [Trello](https://trello.com/c/TAL1jQcB/2692-cixlp1988514tripleociproa-source-image-rejected-none-of-the-signatures-were-accepted-reasons-open-etc-pki-rpm-gpg-rpm-gpg-key-re) - On CIX board in lane `Waiting on external dependencies` * https://bugs.launchpad.net/tripleo/+bug/1987323 - > FIPS + Manila + Ceph Quincy - > ... known issue on the ceph side, ... already merged in master/quincy ... > 17.2.4 should solve this issue, so we'll check during the next promotion. - On CIX board in lane `Waiting on external dependencies` - Not a blocker * https://bugs.launchpad.net/tripleo/+bug/1988810 - Revert https://review.rdoproject.org/r/c/rdo-jobs/+/44788 * https://bugs.launchpad.net/tripleo/+bug/1989197 - > Tempest test `neutron_tempest_plugin.api.test_port_forwardings.PortForwardingTestJSON` fails on `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master` ### Intermittent Failures * https://bugs.launchpad.net/tripleo/+bug/1987616 - > cs9 ovb fs01 clients wallaby is failing with error - 'neutron.agent.dhcp.agent oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply' - no CIX card * https://bugs.launchpad.net/tripleo/+bug/1938987 - periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train - > ERROR keystone.server.flask.request_processing.middleware.auth_context oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (1047, 'WSREP has not yet prepared node for application use') - [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train`](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/bd0a94d/logs/overcloud-controller-2/var/log/containers/keystone/keystone.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train`](https://logserver.rdoproject.org/62/44662/6/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/5753657/logs/overcloud-controller-0/var/log/containers/keystone/keystone.log.txt.gz) * `TASK [freeipa-setup : Run the tripleo-ipa preparation script]` - [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/205c81b/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/f940a55/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby/bfa6f03/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/40/44840/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/57331f2/job-output.txt) * `Connection failure: The read operation timed out` in `TASK [Get current autoholds]` - [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/c3a98c9/job-output.txt) * `Create containers managed by Podman for ... Failed containers: rabbitmq_bootstrap` (or others) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/4926640/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`](https://logserver.rdoproject.org/56/44656/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/007c1da/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [FAILURE in `tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/df357c2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master`](https://logserver.rdoproject.org/57/44657/9/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master/3fb6e42/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) * `FATAL | Wait for puppet host configuration to finish | undercloud | error`... - `TASK [tripleo.operator.tripleo_undercloud_install : undercloud install]` in [job-output.txt](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/job-output.txt) - `puppet-user: Error: Execution of '/bin/dnf -d 0 -e 1 -y install net-snmp' returned 1: Error: Error downloading packages:` in [logs/undercloud/home/zuul/undercloud_install.log.txt.gz](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz) - [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz) * `Exception: Command '['ipmitool', ..., 'status']' timed out after 60 seconds"` during overcloud deploy - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/0a8588f/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz) ### Fixed * https://bugs.launchpad.net/tripleo/+bug/1988347 - fails on c9 master and c9 wallaby - fs064, fs39 and others - intermittent failure aka when rerunning hard enough it will pass eventually! - > `FATAL | try modifying forward dns record | undercloud | error={"changed": false, "msg": "'arecord' not found."}` - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master`](https://logserver.rdoproject.org/openstack-component-security/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master/6ea2096/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master`](https://logserver.rdoproject.org/56/44656/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master/0c48b0f/logs/undercloud/home/zuul/overcloud1_deploy.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-tripleo-master`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-tripleo-master/a0594a2/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-tripleo-master`](https://logserver.rdoproject.org/57/44657/6/check/periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-tripleo-master/8226ab0/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/545f2f1/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master`](https://logserver.rdoproject.org/56/44656/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/40d3797/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master`](https://logserver.rdoproject.org/openstack-component-security/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master/6146852/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) - Ansible pretends that commands `arecord` and `ptrrecord` are while trying to run modules in [role tripleo_ipa_dns](https://opendev.org/x/tripleo-ipa/src/branch/master/tripleo_ipa/roles/tripleo_ipa_dns/tasks/dns.yaml) - [history of job `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master&skip=0) shows e.g. a failure on [2022-08-22](https://logserver.rdoproject.org/openstack-component-security/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master/f8018e3/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz) but subsequent passes - not a legit failure: https://bugs.launchpad.net/tripleo/+bug/1988347/comments/5 ### Downstream: * rhos16.2 on rhel8: * Promotion: 07-sept-2022 * rhos17.1 on rhel8 * Promotion: 08-sept-2022 * rhos17 on rhel9: * Promotion: 08-Sept-2022 * rhos17.1 on rhel9: * Promoted: 09-sept-2022 ### Downstream component: --- ## 2022-09-08 Promotion Status * Downstream Promotion Status + RHOS-16.2 - 07-09-2022 + RHOS-17.0 on RHEL-9 - 04-09-2022 + RHOS-17.1 on RHEL-9 - 07-09-2022 + RHOS-17.1 on RHEL-8 - 07-09-2022 * Upstream Promotion Status + Master - 31-08-2022 + Wallaby CS9 - 05-09-2022 + Wallaby CS8 - 07-09-2022 + Train - 07-09-2022 * Jobs removed to promote content - for network master component - skipped periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master - for wallaby cs8 - periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-wallaby - for cs9 wallaby we might remove fs064 job Bugs: * https://bugs.launchpad.net/tripleo/+bug/1988738 * Octavia ovn driver function create_vip_port wasn't updated with latest octavia changes - Waiting on master promotion * https://bugs.launchpad.net/tripleo/+bug/1988810 - Revert https://review.rdoproject.org/r/c/rdo-jobs/+/44788 * https://trello.com/c/TAL1jQcB/2692-cixlp1988514tripleociproa-source-image-rejected-none-of-the-signatures-were-accepted-reasons-open-etc-pki-rpm-gpg-rpm-gpg-key-re ---