owned this note
owned this note
Published
Linked with GitHub
# Ruck Rover 2022-09-13 to 2022-09-15
###### tags: `ruck_rover`
###### Next RR notes: https://hackmd.io/a92u74hDQhe5afE_LUtbRw
###### Previous RR notes: https://hackmd.io/s4TgnCY-QQGKv2ONxTjOZA
##### ruck: Bhagyashris, rover: Jakob
[RDO Cockpit](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1) / [RHOS Cockpit](http://tripleo-cockpit.lab4.eng.bos.redhat.com)
[RDO Promoter](http://promoter.rdoproject.org/promoter_logs/) / [RHOS Promoter](http://10.0.110.143/promoter_logs/)
[OpenStack Program Meeting 2022](
https://docs.google.com/document/d/1n6ArkMh68R9zivjlyGbpedkggk1wMwEIcrMZSN2uIjc/edit)
Zuul Status:
* [opendev.org:openstack](https://zuul.opendev.org/t/openstack/status/)
* [rdoproject.org:rdoproject.org](https://review.rdoproject.org/zuul/status)
* [redhat.com:tripleo-ci-internal](https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status)
---
## 2022-09-15
### TLDR
* Upstream
- **TODO**
* Downstream
- **TODO**
### Reruns and Investigations:
(crossed out := known bugs recorded above)
**NOTE:** Watch for running `testproject` jobs on
https://review.rdoproject.org/zuul/status and
https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status.
#### Integration Lines
* [centos-9 master](
http://dashboard-ci.tripleo.org/d/mde16Npnz/centos-9-and-centos-8-comparison-master)
- last_promotion=2022-09-13
- [hash `8c71d10c1dacd8b2b5fc59dbdd753186`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_agg_detail.html?ref_hash=8c71d10c1dacd8b2b5fc59dbdd753186)
+ [`periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master):
* [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master/8ffe44e)
+ [`periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master):
* [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master/69063f9)
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master):
* [Overcloud stack: FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/adf4075)
+ [`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master):
* [Tempest tests FAILED. Reason: unknown ](https://sf.hosted.upshift.rdu2.redhat.com/logs/11/44711/118/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/7fba01d)
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master):
* [Overcloud stack: FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master/8a99ecc)
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master):
* [N/A](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master/588602f)
* [centos-9 wallaby](
http://dashboard-ci.tripleo.org/d/NRJeqYBnz/centos-9-and-centos-8-comparison-wallaby)
- last_promotion=2022-09-13
- [hash `164135bf6866a2568241e186f56b903b`](https://trunk.rdoproject.org/api-centos9-wallaby/api/civotes_agg_detail.html?ref_hash=164135bf6866a2568241e186f56b903b)
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby):
* [Overcloud stack: FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/8679de7)
+ [`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-wallaby):
* [Tempest tests FAILED. Reason: unknown ](https://sf.hosted.upshift.rdu2.redhat.com/logs/11/44711/117/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-wallaby/bbd68f1)
+ [`periodic-tripleo-ci-centos-9-standalone-full-tempest-api-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-api-wallaby):
* [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-wallaby/a630f17)
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby):
* [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/ed97c89)
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby):
* [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/f4cd59b)
+ [`periodic-tripleo-ci-centos-9-scenario003-standalone-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario003-standalone-wallaby):
* [N/A](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario003-standalone-wallaby/e700966)
+ [`periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-wallaby):
* [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-wallaby/6df3180)
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby):
* [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/4d47b44)
* [centos-8 wallaby](
http://dashboard-ci.tripleo.org/d/NRJeqYBnz/centos-9-and-centos-8-comparison-wallaby)
- last_promotion=2022-09-14
- [hash `5b62e82eebd71259ed086d27d7a2abae`](https://trunk.rdoproject.org/api-centos8-wallaby/api/civotes_agg_detail.html?ref_hash=5b62e82eebd71259ed086d27d7a2abae)
* [centos-8 train](
http://dashboard-ci.tripleo.org/d/mOvYIiOMk/component-pipeline-train)
- last_promotion=2022-09-12
- [hash `54e3db5e5134970e993eeb7d124da84c`](https://trunk.rdoproject.org/api-centos8-train/api/civotes_agg_detail.html?ref_hash=54e3db5e5134970e993eeb7d124da84c)
+ [`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train):
* [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/65/31165/92/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/f55e2c7)
* [rhel-9 osp17-1](
http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/bSwsg0WVz/rhel9-rhos17-1-full-component-pipeline)
- last_promotion=2022-09-14
- [hash `fb7fef6ff353625dadab13dfa1c2c80e`](https://osp-trunk.hosted.upshift.rdu2.redhat.com/api-rhel9-osp17-1/api/civotes_agg_detail.html?ref_hash=fb7fef6ff353625dadab13dfa1c2c80e)
* [rhel-9 osp17](
http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/lF7RUpsnk/rhel9-rhos17-full-component-pipeline)
- last_promotion=2022-09-11
- [hash `0990788a06ae1c3505956399b88fd630`](https://osp-trunk.hosted.upshift.rdu2.redhat.com/api-rhel9-osp17/api/civotes_agg_detail.html?ref_hash=0990788a06ae1c3505956399b88fd630)
* [rhel-8 osp16-2](
http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/KyHCwLHMk/rhos-16-2-full-component-pipeline)
- last_promotion=2022-09-13
- [hash `546f5261237fd7e18a39ce31ae980932`](https://osp-trunk.hosted.upshift.rdu2.redhat.com/api-rhel8-osp16-2/api/civotes_agg_detail.html?ref_hash=546f5261237fd7e18a39ce31ae980932)
+ [`periodic-tripleo-build-containers-rhel-8-internal-build-push-upload-rhos-16.2`](
https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-build-containers-rhel-8-internal-build-push-upload-rhos-16.2):
* [N/A](https://sf.hosted.upshift.rdu2.redhat.com/logs/00/422400/14/check/periodic-tripleo-build-containers-rhel-8-internal-build-push-upload-rhos-16.2/f09c1b4)
#### Component Lines
* [centos-9 master](
http://dashboard-ci.tripleo.org/d/M1DYSidGz/component-pipeline)
- baremetal
+ last_promotion=2022-09-13
+ [hash `a99bf274e4baec8e585bc9979e492bb8d85d17b5`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_detail.html?commit_hash=a99bf274e4baec8e585bc9979e492bb8d85d17b5&distro_hash=d7d161658d469b9211bb2b8cfabc9aa4fca9ed68)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master):
- [Overcloud stack: FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-component-baremetal/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master/f5ac18b)
- cinder
+ last_promotion=2022-09-14
+ [hash `87205a242bbf1c0c39490a1d518d6918d08addca`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_detail.html?commit_hash=87205a242bbf1c0c39490a1d518d6918d08addca&distro_hash=f8434e1ff5730482ab08ad641743a491c74646b4)
* [`periodic-tripleo-ci-centos-9-scenario004-standalone-cinder-master`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario004-standalone-cinder-master):
- [N/A](https://logserver.rdoproject.org/openstack-component-cinder/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario004-standalone-cinder-master/8a11593)
- common
+ last_promotion=2022-09-14
+ [hash `ee5d8644c328e850c561c99b2098aeba2bf1b471`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_detail.html?commit_hash=ee5d8644c328e850c561c99b2098aeba2bf1b471&distro_hash=e078d66a731739522e3da5dbb8786aa4496740db)
- compute
+ last_promotion=2022-09-07
+ [hash `299c898cf4af1d96a4c683305d3c262285eccdaf`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_detail.html?commit_hash=299c898cf4af1d96a4c683305d3c262285eccdaf&distro_hash=2af3f376e5c5ad8caa7e307511b550bcdfb93d1d)
* [`periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master):
- [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/57/44657/18/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master/d4674de)
- security
+ last_promotion=2022-09-13
+ [hash `44cdbc68658d8fe3013d1dd253a268bec343c087`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_detail.html?commit_hash=44cdbc68658d8fe3013d1dd253a268bec343c087&distro_hash=8cbe449b641de048de115303a8a4e9226068befa)
- tempest
+ last_promotion=2022-09-05
+ [hash `b39bbcc0bc5be39cc4d9e3cf2addf1924c8df49a`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_detail.html?commit_hash=b39bbcc0bc5be39cc4d9e3cf2addf1924c8df49a&distro_hash=c770ffa7eab8d0e5ad9c2a476b166f3e3175fb89)
- tripleo
+ last_promotion=2022-09-14
+ [hash `948a5f23d866e2d30a5d4c2e723a63d1aae637e6`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_detail.html?commit_hash=948a5f23d866e2d30a5d4c2e723a63d1aae637e6&distro_hash=d7b1b549640cd2d447542d79bbefdadf4d271d90)
* [centos-9 wallaby](
http://dashboard-ci.tripleo.org/d/b0e2jBLnk/component-pipeline-wallaby-centos9)
- tempest
+ last_promotion=2022-09-07
+ [hash `d5fc8ad86bb895ebf59ff73bdcf36a9a6b7eff4f`](https://trunk.rdoproject.org/api-centos9-wallaby/api/civotes_detail.html?commit_hash=d5fc8ad86bb895ebf59ff73bdcf36a9a6b7eff4f&distro_hash=048d9d649c588abdc0b659ced95fdc6ea2d007f0)
* [`periodic-tripleo-ci-centos-9-standalone-full-tempest-scenario-tempest-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-scenario-tempest-wallaby):
- [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-component-tempest/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-scenario-tempest-wallaby/3fcb900)
- tripleo
+ last_promotion=2022-09-12
+ [hash `39be6cfe5cc0e18bbea85705f593b7608cadd0a6`](https://trunk.rdoproject.org/api-centos9-wallaby/api/civotes_detail.html?commit_hash=39be6cfe5cc0e18bbea85705f593b7608cadd0a6&distro_hash=bfb6fe68901ad22fe9ff2addedeaead32889acc4)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby):
- [Tempest tests FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/81c8259)
* [centos-8 wallaby](
http://dashboard-ci.tripleo.org/d/TWhcr9_Mk/component-pipeline-wallaby)
- network
+ last_promotion=2022-09-09
+ [hash `b125402309516d25e40113f3963d1bdf0a25421b`](https://trunk.rdoproject.org/api-centos8-wallaby/api/civotes_detail.html?commit_hash=b125402309516d25e40113f3963d1bdf0a25421b&distro_hash=d1361717722f81c13cf3874411ee1cbbe775d169)
* [`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-wallaby):
- [MAC conflict between hosts. Overcloud stack: FAILED. Reason: infra ](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-wallaby/e403b0f)
- tripleo
+ last_promotion=2022-09-13
+ [hash `39be6cfe5cc0e18bbea85705f593b7608cadd0a6`](https://trunk.rdoproject.org/api-centos8-wallaby/api/civotes_detail.html?commit_hash=39be6cfe5cc0e18bbea85705f593b7608cadd0a6&distro_hash=bfb6fe68901ad22fe9ff2addedeaead32889acc4)
* [`periodic-tripleo-ci-centos-8-standalone-tripleo-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-tripleo-wallaby):
- [N/A](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-tripleo-wallaby/46c6dc5)
* [`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-wallaby):
- [Overcloud stack: FAILED. Reason: unknown ](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/579a59f)
* [centos-8 train](
http://dashboard-ci.tripleo.org/d/mOvYIiOMk/component-pipeline-train)
- tripleo
+ last_promotion=2022-09-10
+ [hash `f7fa1d52a7f5d7ae868bc6a57da5e7cae1c84748`](https://trunk.rdoproject.org/api-centos8-train/api/civotes_detail.html?commit_hash=f7fa1d52a7f5d7ae868bc6a57da5e7cae1c84748&distro_hash=90903973080e5443bd999c959ea13f406c63d5a2)
* [`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train`](
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train):
- [Timeout. Reason: unknown ](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train/b804f15)
* [rhel-9 osp17-1](
http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/bSwsg0WVz/rhel9-rhos17-1-full-component-pipeline)
* [rhel-9 osp17](
http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/lF7RUpsnk/rhel9-rhos17-full-component-pipeline)
* [rhel-8 osp16-2](
http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/KyHCwLHMk/rhos-16-2-full-component-pipeline)
### Known Bugs
* https://bugs.launchpad.net/tripleo/+bug/1960355
- > Tempest tests test_oauth1_tokens.OAUTH1TokensTest are failing on tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-(master|wallaby)
- affecting OSP17.1 as well
* https://bugs.launchpad.net/tripleo/+bug/1987092 :warning: **BLOCKER** :warning:
- Pacemaker performance causes intermittent galera issues in loaded CI env
- jobs fail on `Check Keystone public endpoint status` or in other places
- [zuul/builds](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-validation-master&skip=0)
- [affecting non-validation jobs such as periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby](https://bugs.launchpad.net/tripleo/+bug/1987092/comments/18)
* https://bugzilla.redhat.com/2105408 :warning: **BLOCKER** :warning:
- > RHOSP16.2 - Octavia tempest test "test_load_balancer_ipv4_CRUD" fails due to a kernel stacktrace (in a nested KVM setup)
- > reproducible only on RHEL 8.2 and RHEL 8.4 ... RHEL 8.6 is unaffected. So, please update the affected CI systems to RHEL 8.6.
- [latest status is tracked in cix card](https://trello.com/c/msIodUbD/2616-cixbz2105408osp162rhel8octavia-tempest-test-loadbalancerscenariotesttestloadbalanceripv4crud-failing-on-load-balancer-creation-t)
- blocking integration lines of c9 master (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master`) and c9 wallaby (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-wallaby`)
- [workaround (specifying `cpu_models` in libvirt)](
https://code.engineering.redhat.com/gerrit/c/openstack-tripleo-heat-templates/+/425806/) is
[currently being tested](https://code.engineering.redhat.com/gerrit/c/testproject/+/425432/)
* https://bugs.launchpad.net/tripleo/+bug/1988514
- [Trello](https://trello.com/c/TAL1jQcB/2692-cixlp1988514tripleociproa-source-image-rejected-none-of-the-signatures-were-accepted-reasons-open-etc-pki-rpm-gpg-rpm-gpg-key-re)
- On CIX board in lane `Waiting on external dependencies`
* https://bugs.launchpad.net/tripleo/+bug/1987323
- > FIPS + Manila + Ceph Quincy
- > ... known issue on the ceph side, ... already merged in master/quincy ...
> 17.2.4 should solve this issue, so we'll check during the next promotion.
- On CIX board in lane `Waiting on external dependencies`
- Not a blocker
* https://bugs.launchpad.net/tripleo/+bug/1988810
- Revert https://review.rdoproject.org/r/c/rdo-jobs/+/44788
* https://bugs.launchpad.net/tripleo/+bug/1989197
- > Tempest test `neutron_tempest_plugin.api.test_port_forwardings.PortForwardingTestJSON` fails on `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master`
* https://bugs.launchpad.net/tripleo/+bug/1989341
- > Tempest tests "tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON" etc. fail on {periodic-,}tripleo-ci-centos-8-9-multinode-mixed-os due to ovn controller/north version mismatch
* https://bugs.launchpad.net/tripleo/+bug/1989247
- > overcloud deploy fails with `sudo: unable to send audit message: Operation not permitted`
- > `Create containers managed by Podman for ... Failed containers: rabbitmq_bootstrap` (or others)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`](https://logserver.rdoproject.org/56/44656/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/007c1da/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master`](https://logserver.rdoproject.org/57/44657/9/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master/3fb6e42/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* https://bugs.launchpad.net/tripleo/+bug/1989452
- > multiple periodic integration jobs fail configure-mirrors - Failed to connect to mirrors.centos.org port 443: No route to host
* https://bugs.launchpad.net/tripleo/+bug/1989606
- > container creation during overcloud deploy fails on c9/c8 master fs1 with "You have to remove that container to be able to reuse that name.: that name is already in use"
### Intermittent Failures
* https://bugs.launchpad.net/tripleo/+bug/1987616
- > cs9 ovb fs01 clients wallaby is failing with error - 'neutron.agent.dhcp.agent oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply'
- no CIX card
* https://bugs.launchpad.net/tripleo/+bug/1938987
- periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train
- > ERROR keystone.server.flask.request_processing.middleware.auth_context oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (1047, 'WSREP has not yet prepared node for application use')
- [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train`](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/bd0a94d/logs/overcloud-controller-2/var/log/containers/keystone/keystone.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train`](https://logserver.rdoproject.org/62/44662/6/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/5753657/logs/overcloud-controller-0/var/log/containers/keystone/keystone.log.txt.gz)
* `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`
- [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/205c81b/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/f940a55/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby/bfa6f03/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/40/44840/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/57331f2/job-output.txt)
* `Connection failure: The read operation timed out` in `TASK [Get current autoholds]`
- [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/c3a98c9/job-output.txt)
* `Create containers managed by Podman for ... Failed containers: rabbitmq_bootstrap` (or others)
- > this has different root causes, watch out for log details!
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/4926640/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
- [FAILURE in `tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/df357c2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* `FATAL | Wait for puppet host configuration to finish | undercloud | error`...
- `TASK [tripleo.operator.tripleo_undercloud_install : undercloud install]` in [job-output.txt](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/job-output.txt)
- `puppet-user: Error: Execution of '/bin/dnf -d 0 -e 1 -y install net-snmp' returned 1: Error: Error downloading packages:` in [logs/undercloud/home/zuul/undercloud_install.log.txt.gz](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz)
- [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz)
* `Exception: Command '['ipmitool', ..., 'status']' timed out after 60 seconds"` during overcloud deploy
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/0a8588f/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz)
### Fixed
### Downstream:
* rhos16.2 on rhel8:
* Promotion: 13-sept-2022
* In todays run conatiner build job failed so re-running the job here: https://code.engineering.redhat.com/gerrit/c/testproject/+/422400
* container build job failing due to this change https://review.opendev.org/c/openstack/tripleo-ci/+/850736
* job passed on revert change https://review.opendev.org/c/openstack/tripleo-ci/+/857776 job result: periodic-tripleo-build-containers-rhel-8-internal-build-push-upload-rhos-16.2_1 https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/39d84ef71b86408aa433b5edbd452acc : SUCCESS in 1h 08m 23s
* rhos17.1 on rhel8
* Promotion: 14-sept-2022
* rhos17 on rhel9:
* Promotion: 11-Sept-2022
* Note: we don't have new content to promote.
* rhos17.1 on rhel9:
* Promoted: 15-sept-2022
* Waiting for jekins jobs result, integration line is GREEN
### Downstream component:
---
## 2022-09-14
### Notes
* doug: hi o/ - we merged a small change on sshd_config that affects centos-9 images (https://softwarefactory-project.io/r/c/config/+/26113) - note that this image isn't build on daily basis anymore, so our jobs won't get this update unless we update the base image and request rhos-infra team to unpause the build. Just in case you need a new image, keep an eye on related ssh issues with missing kex alghoritm curve25519.
### Reminder:
Please update https://docs.engineering.redhat.com/display/PRODCHAIN/2022-09-12+Meeting+Notes for the program call. wallaby c8, c9, train, 16.2, 17 , 17.1 on rhel-9 and rhel-8
### Reruns and Investigations:
(crossed out := known bugs recorded above)
**NOTE:** Watch for running `testproject` jobs on https://review.rdoproject.org/zuul/status and
https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status.
* [Promotions](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1)
- C9 Master
+ last_promotion: 2022-09-13
+ [hash `573ed7f477dfe3ed338408f28dac7035`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_agg_detail.html?ref_hash=573ed7f477dfe3ed338408f28dac7035)
* ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master)~~:
- ~~[TODO](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/222e953)~~
* [`periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master):
- [Overcloud deploy failed with `ERROR: Can't run container glance_api_db_sync`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master/6706077/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* [Overcloud stack: FAILED. Reason: `ERROR: Can't run container rabbitmq_bootstrap`](https://logserver.rdoproject.org/56/44656/21/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_2comp-featureset020-master/81b0212/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master):
- [undercloud install failed with `ensure apache is installed...Failed to download packages: centos-logos-httpd-***el9.noarch: Cannot download, all mirrors were already tried without success`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/6b6ada3/logs/undercloud/home/zuul/undercloud_install.log.txt.gz)
* [Overcloud stack: FAILED. Reason: `ConnectionRefusedError: [Errno 111] Connection refused` ](https://logserver.rdoproject.org/56/44656/21/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/c3fb2a5/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master):
- [intermittent failure `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master/6265a3a/job-output.txt)
* [Overcloud stack: FAILED. Reason: `ERROR: Can't run container nova_db_sync`](https://logserver.rdoproject.org/56/44656/21/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master/7ed641e/logs/undercloud/home/zuul/overcloud1_deploy.log.txt.gz)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master):
- [Overcloud deploy failed with `ERROR: Can't run container mysql_wait_bundle`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master/4d0da94/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* [Tempest tests FAILED](https://logserver.rdoproject.org/56/44656/21/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master/29df173/logs/undercloud/var/log/tempest/stestr_results.html.gz)
* ~~[`periodic-tripleo-ci-centos-9-scenario001-standalone-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario001-standalone-master)~~:
- ~~[TODO](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario001-standalone-master/a2c9332)~~
* [`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master):
- [LP#2105408?](https://sf.hosted.upshift.rdu2.redhat.com/logs/11/44711/109/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/0e4c843/logs/undercloud/var/log/tempest/stestr_results.html)
* [LP#2105408?](https://sf.hosted.upshift.rdu2.redhat.com/logs/11/44711/109/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/0e4c843/logs/undercloud/var/log/tempest/stestr_results.html)
* [reruns 1](https://review.rdoproject.org/r/c/testproject/+/44656)
* [reruns 2](https://code.engineering.redhat.com/gerrit/c/testproject/+/426429) (internal)
- C9 Wallaby
+ last_promotion: 2022-09-13
+ [hash `84acb1bac5b6c640738a43e63c532c2c`](https://trunk.rdoproject.org/api-centos9-wallaby/api/civotes_agg_detail.html?ref_hash=84acb1bac5b6c640738a43e63c532c2c)
* [`periodic-tripleo-ci-centos-9-scenario003-standalone-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario003-standalone-wallaby):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario003-standalone-wallaby/6329674)
* [`periodic-tripleo-ci-centos-9-standalone-full-tempest-api-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-api-wallaby):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-wallaby/9c8a8eb)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/11873f7)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/a9756ff)
* ~~[`periodic-tripleo-ci-centos-8-9-multinode-mixed-os`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-9-multinode-mixed-os)~~ (not in criteria):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1-cs8/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-9-multinode-mixed-os/a88b2c9)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/3fda6a3)
* ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-internal-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-internal-wallaby)~~ (not in criteria):
- [TODO](https://sf.hosted.upshift.rdu2.redhat.com/logs/37/44737/23/check-rdo/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-internal-wallaby/79d5853)
* ~~[`periodic-tripleo-ci-centos-9-8-multinode-mixed-os`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-8-multinode-mixed-os)~~ (not in criteria):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-8-multinode-mixed-os/daeab46)
* ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-internal-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-internal-wallaby)~~ (not in criteria):
- [TODO](https://sf.hosted.upshift.rdu2.redhat.com/logs/38/44738/23/check-rdo/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-internal-wallaby/4a8d7b4)
* ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby)~~ (not in criteria):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby/7ac5426)
* [reruns 1](https://review.rdoproject.org/r/c/testproject/+/44661)
- C8 Wallaby
+ last_promotion: 2022-09-11
+ [hash `5b62e82eebd71259ed086d27d7a2abae`](https://trunk.rdoproject.org/api-centos8-wallaby/api/civotes_agg_detail.html?ref_hash=5b62e82eebd71259ed086d27d7a2abae)
* [`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-wallaby):
- [`TASK [run-test : run toci_gate_test.sh]` failed with `HTTPConnectionPool(host='mirror.regionone.ibm-bm2-nodepool.rdoproject.org', port=80): Read timed out.`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1-cs8/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-wallaby/5affdf9/job-output.txt)
- [overcloud deploy failed with `Error: 'rabbitmq-bundle' already exists. Too many tries`](https://logserver.rdoproject.org/83/44983/1/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-wallaby/8aad894/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
- [overcloud deploy failed with `tripleoclient.exceptions.HeatPodMessageQueueException: Message queue for ephemeral heat not created in time`](https://logserver.rdoproject.org/83/44983/1/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-wallaby/6700c1a/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* [reruns 1](https://review.rdoproject.org/r/c/testproject/+/44983)
- C8 Train
+ last_promotion: 2022-09-12
+ [hash `54e3db5e5134970e993eeb7d124da84c`](https://trunk.rdoproject.org/api-centos8-train/api/civotes_agg_detail.html?ref_hash=54e3db5e5134970e993eeb7d124da84c)
* [`periodic-tripleo-ci-centos-8-standalone-full-tempest-api-train`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-full-tempest-api-train):
- [tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-api-train/1844f24/logs/undercloud/var/log/tempest/stestr_results.html)
- [other (?) tempest failures](https://logserver.rdoproject.org/62/44662/13/check/periodic-tripleo-ci-centos-8-standalone-full-tempest-api-train/40a4f2d/logs/undercloud/var/log/tempest/stestr_results.html.gz)
* [reruns](https://review.rdoproject.org/r/c/testproject/+/44662)
* hash: 4e3db5e5134970e993eeb7d124da84c missing on fs035 - rerun https://review.rdoproject.org/r/c/testproject/+/31165
- RHOS-17.1 on RHEL-9 **Refer to Bhagyashris' notes below**
- RHOS-17.1 on RHEL-8 **Refer to Bhagyashris' notes below**
- RHOS-17.0 on RHEL-9 **Refer to Bhagyashris' notes below**
- RHOS-16.2 on RHEL-8 **Refer to Bhagyashris' notes below**
#### Component Lines
* [C9 Master](http://dashboard-ci.tripleo.org/d/2tivP9BWz/component-pipeline?orgId=1):
- baremetal
+ last_promotion: 2022-09-13
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master):
* [TODO](https://logserver.rdoproject.org/openstack-component-baremetal/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master/c0ceb4b)
- clients
+ last_promotion: 2022-09-09
+ ~~[`periodic-tripleo-ci-centos-9-undercloud-upgrade-clients-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-undercloud-upgrade-clients-master)~~ (not in criteria):
* [TODO](https://logserver.rdoproject.org/openstack-component-clients/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-upgrade-clients-master/2f6706b)
- common
+ last_promotion: 2022-09-11
+ ~~[`periodic-tripleo-ci-centos-9-undercloud-upgrade-common-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-undercloud-upgrade-common-master)~~ (not in criteria):
* [TODO](https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-upgrade-common-master/03aa0d4)
- compute
+ last_promotion: 2022-09-07
+ [`periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master):
* [TODO](https://logserver.rdoproject.org/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-compute-master/5c6b3ab)
- manila
+ last_promotion: 2022-09-12
+ [`periodic-tripleo-ci-centos-9-scenario004-standalone-manila-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario004-standalone-manila-master):
* [TODO](https://logserver.rdoproject.org/openstack-component-manila/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario004-standalone-manila-master/74eace2)
- tempest
+ last_promotion: 2022-09-05
+ ~~[`periodic-tripleo-ci-centos-9-standalone-tempest-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-tempest-master)~~:
* ~~[TODO](https://logserver.rdoproject.org/openstack-component-tempest/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-tempest-master/4540c6d)~~
+ [`periodic-tripleo-ci-centos-9-standalone-full-tempest-api-tempest-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-api-tempest-master):
* [TODO](https://logserver.rdoproject.org/openstack-component-tempest/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-tempest-master/8785c6a)
- tripleo
+ last_promotion: 2022-09-12
+ ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-master)~~:
* ~~[TODO](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-master/00f4c19)~~
- [reruns](https://review.rdoproject.org/r/c/testproject/+/44657)
* [C9 Wallaby](http://dashboard-ci.tripleo.org/d/UcMR8py7z/component-pipeline-wallaby-centos9?orgId=1):
- common
+ last_promotion: 2022-09-10
+ ~~[`periodic-tripleo-ci-centos-9-8-multinode-mixed-os-common-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-8-multinode-mixed-os-common-wallaby)~~ (not in criteria):
* [TODO](https://logserver.rdoproject.org/84/44684/7/check/periodic-tripleo-ci-centos-9-8-multinode-mixed-os-common-wallaby/5fa2da2)
- network
+ last_promotion: 2022-09-13
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby):
* [TODO](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-wallaby/62f7c50)
- tempest
+ last_promotion: 2022-09-07
+ [`periodic-tripleo-ci-centos-9-standalone-full-tempest-api-tempest-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-api-tempest-wallaby):
* [TODO](https://logserver.rdoproject.org/openstack-component-tempest/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-tempest-wallaby/b7d82af)
- tripleo
+ last_promotion: 2022-09-12
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby):
* [TODO](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/0747d5f)
- [reruns](https://review.rdoproject.org/r/c/testproject/+/44658)
* [C8 Wallaby](http://dashboard-ci.tripleo.org/d/ZLvFbT9Mz/component-pipeline-wallaby?orgId=1):
- network
+ last_promotion: 2022-09-09
+ [`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-wallaby):
* [TODO](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-wallaby/b4d2ca6)
- tripleo
+ last_promotion: 2022-09-13
+ [`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-wallaby):
* [TODO](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/fad474c)
- [reruns](https://review.rdoproject.org/r/c/testproject/+/44659)
* [C8 Train](http://dashboard-ci.tripleo.org/d/aJeQpzVGz/component-pipeline-train?orgId=1):
- network
+ last_promotion: 2022-09-09
+ ~~[`periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train)~~ (not in criteria):
* [TODO](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train/e7f86e0)
- tripleo
+ last_promotion: 2022-09-10
+ [`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train):
* [TODO](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train/9f349bb)
- [reruns](https://review.rdoproject.org/r/c/testproject/+/44690)
* [RHOS-17.0 on RHEL-9](http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/lF7RUpsnk/rhel9-rhos17-full-component-pipeline?orgId=1):
* **Refer to Bhagyashris' notes below**
* [RHOS-16.2 on RHEL-8](http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/KyHCwLHMk/rhos-16-2-full-component-pipeline?orgId=1):
* **Refer to Bhagyashris' notes below**
### Downstream:
* rhos16.2 on rhel8:
* Promotion: 13-sept-2022
* blocked due to fule-tempest api , scenario and fs020 re-running jobs here: https://code.engineering.redhat.com/gerrit/c/testproject/+/422400
* rhos17.1 on rhel8
* Promotion: 14-sept-2022
* buildimage overcloud full and tempest api job failed re-running here: https://code.engineering.redhat.com/gerrit/c/testproject/+/424992
* rhos17 on rhel9:
* Promotion: 14-Sept-2022
* Note: we don't have new content to promote.
* rhos17.1 on rhel9:
* Promoted: 12-sept-2022
*
### Downstream component:
### Known Bugs
* https://bugs.launchpad.net/tripleo/+bug/1960355
- > Tempest tests test_oauth1_tokens.OAUTH1TokensTest are failing on tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-(master|wallaby)
- affecting OSP17.1 as well
* https://bugs.launchpad.net/tripleo/+bug/1987092 :warning: **BLOCKER** :warning:
- Pacemaker performance causes intermittent galera issues in loaded CI env
- jobs fail on `Check Keystone public endpoint status` or in other places
- [zuul/builds](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-validation-master&skip=0)
- [affecting non-validation jobs such as periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby](https://bugs.launchpad.net/tripleo/+bug/1987092/comments/18)
* https://bugzilla.redhat.com/2105408 :warning: **BLOCKER** :warning:
- > RHOSP16.2 - Octavia tempest test "test_load_balancer_ipv4_CRUD" fails due to a kernel stacktrace (in a nested KVM setup)
- > reproducible only on RHEL 8.2 and RHEL 8.4 ... RHEL 8.6 is unaffected. So, please update the affected CI systems to RHEL 8.6.
- [latest status is tracked in cix card](https://trello.com/c/msIodUbD/2616-cixbz2105408osp162rhel8octavia-tempest-test-loadbalancerscenariotesttestloadbalanceripv4crud-failing-on-load-balancer-creation-t)
- blocking integration lines of c9 master (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master`) and c9 wallaby (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-wallaby`)
- [workaround (specifying `cpu_models` in libvirt)](
https://code.engineering.redhat.com/gerrit/c/openstack-tripleo-heat-templates/+/425806/) is
[currently being tested](https://code.engineering.redhat.com/gerrit/c/testproject/+/425432/)
* https://bugs.launchpad.net/tripleo/+bug/1988514
- [Trello](https://trello.com/c/TAL1jQcB/2692-cixlp1988514tripleociproa-source-image-rejected-none-of-the-signatures-were-accepted-reasons-open-etc-pki-rpm-gpg-rpm-gpg-key-re)
- On CIX board in lane `Waiting on external dependencies`
* https://bugs.launchpad.net/tripleo/+bug/1987323
- > FIPS + Manila + Ceph Quincy
- > ... known issue on the ceph side, ... already merged in master/quincy ...
> 17.2.4 should solve this issue, so we'll check during the next promotion.
- On CIX board in lane `Waiting on external dependencies`
- Not a blocker
* https://bugs.launchpad.net/tripleo/+bug/1988810
- Revert https://review.rdoproject.org/r/c/rdo-jobs/+/44788
* https://bugs.launchpad.net/tripleo/+bug/1989197
- > Tempest test `neutron_tempest_plugin.api.test_port_forwardings.PortForwardingTestJSON` fails on `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master`
* https://bugs.launchpad.net/tripleo/+bug/1989341
- > Tempest tests "tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON" etc. fail on {periodic-,}tripleo-ci-centos-8-9-multinode-mixed-os due to ovn controller/north version mismatch
* https://bugs.launchpad.net/tripleo/+bug/1989247
- > overcloud deploy fails with `sudo: unable to send audit message: Operation not permitted`
- > `Create containers managed by Podman for ... Failed containers: rabbitmq_bootstrap` (or others)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`](https://logserver.rdoproject.org/56/44656/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/007c1da/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master`](https://logserver.rdoproject.org/57/44657/9/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master/3fb6e42/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* https://bugs.launchpad.net/tripleo/+bug/1989452
- > multiple periodic integration jobs fail configure-mirrors - Failed to connect to mirrors.centos.org port 443: No route to host
* https://bugs.launchpad.net/tripleo/+bug/1989606
- > container creation during overcloud deploy fails on c9/c8 master fs1 with "You have to remove that container to be able to reuse that name.: that name is already in use"
### Intermittent Failures
* https://bugs.launchpad.net/tripleo/+bug/1987616
- > cs9 ovb fs01 clients wallaby is failing with error - 'neutron.agent.dhcp.agent oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply'
- no CIX card
* https://bugs.launchpad.net/tripleo/+bug/1938987
- periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train
- > ERROR keystone.server.flask.request_processing.middleware.auth_context oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (1047, 'WSREP has not yet prepared node for application use')
- [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train`](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/bd0a94d/logs/overcloud-controller-2/var/log/containers/keystone/keystone.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train`](https://logserver.rdoproject.org/62/44662/6/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/5753657/logs/overcloud-controller-0/var/log/containers/keystone/keystone.log.txt.gz)
* `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`
- [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/205c81b/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/f940a55/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby/bfa6f03/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/40/44840/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/57331f2/job-output.txt)
* `Connection failure: The read operation timed out` in `TASK [Get current autoholds]`
- [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/c3a98c9/job-output.txt)
* `Create containers managed by Podman for ... Failed containers: rabbitmq_bootstrap` (or others)
- > this has different root causes, watch out for log details!
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/4926640/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
- [FAILURE in `tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/df357c2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* `FATAL | Wait for puppet host configuration to finish | undercloud | error`...
- `TASK [tripleo.operator.tripleo_undercloud_install : undercloud install]` in [job-output.txt](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/job-output.txt)
- `puppet-user: Error: Execution of '/bin/dnf -d 0 -e 1 -y install net-snmp' returned 1: Error: Error downloading packages:` in [logs/undercloud/home/zuul/undercloud_install.log.txt.gz](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz)
- [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz)
* `Exception: Command '['ipmitool', ..., 'status']' timed out after 60 seconds"` during overcloud deploy
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/0a8588f/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz)
### Fixed
---
## 2022-09-13
### Reruns and Investigations:
(crossed out := known bugs recorded above)
**NOTE:** Watch for running `testproject` jobs on https://review.rdoproject.org/zuul/status and
https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status.
* [Promotions](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1)
- C9 Master
+ last_promotion: 2022-09-09
+ [hash `71abe3aa5c8ad3da6d894dc93d9c0273`](https://trunk.rdoproject.org/api-centos9-master-uc/api/civotes_agg_detail.html?ref_hash=71abe3aa5c8ad3da6d894dc93d9c0273)
* ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master)~~:
- ~~[overcloud deploy failed with `Clean up legacy Cinder keystone catalog entries`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master/33e26a7/logs/undercloud/home/zuul/overcloud1_deploy.log.txt.gz)~~
- ~~[intermittent failure `TASK [freeipa-setup : Deploy FreeIPA]`](https://logserver.rdoproject.org/56/44656/19/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-master/e25c26b/job-output.txt)~~
* [`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master):
- [tempest failures](https://sf.hosted.upshift.rdu2.redhat.com/logs/11/44711/100/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/73e8876/logs/undercloud/var/log/tempest/stestr_results.html)
- [other tempest failures](https://sf.hosted.upshift.rdu2.redhat.com/logs/29/426429/5/check/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/57896e7/logs/undercloud/var/log/tempest/stestr_results.html)
* ~~[`periodic-tripleo-ci-centos-9-standalone-full-tempest-scenario-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-scenario-master)~~:
- ~~[other tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-scenario-master/ec9dfc7/logs/undercloud/var/log/tempest/stestr_results.html.gz)~~
* ~~[`periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master)~~:
- ~~[overcloud deploy failed with `tripleoclient.exceptions.HeatPodMessageQueueException: Message queue for ephemeral heat not created in time`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master/a2ace6f/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)~~
- ~~[undercloud install failed in `Manage package | undercloud` with `Failed to download packages: double-conversion-3.1.5-6.el9.x86_64: Cannot download, all mirrors were already tried without success`](https://logserver.rdoproject.org/56/44656/19/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-master/825bf9b/logs/undercloud/home/zuul/undercloud_install.log.txt.gz)~~
* ~~[`periodic-tripleo-ci-centos-9-scenario010-kvm-standalone-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario010-kvm-standalone-master)~~ (not in criteria):
- [failed in `TASK [standalone : Write containers-prepare-parameters.yaml]` with `Failed to connect to the host via ssh: kex_exchange_identification: Connection closed by remote host`](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario010-kvm-standalone-master/47f8263)
* ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm-5`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm-5)~~ (not in criteria):
- [overcloud deploy failed with `ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud ConnectionRefusedError: [Errno 111] Connection refused`](https://logserver.rdoproject.org/88/44388/10/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm-5/50185ff/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* ~~[`periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master)~~:
- ~~[other² tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master/4022186/logs/undercloud/var/log/tempest/stestr_results.html.gz)~~
- ~~[standalone deploy failed in `ensure apache is installed` with `Failed to download packages: apr-1.7.0-11.el9.x86_64: Cannot download, all mirrors were already tried without success`](https://logserver.rdoproject.org/56/44656/19/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master/d70465f/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz)~~
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master):
- [other³ tempest failures](https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/d0f8795/logs/undercloud/var/log/tempest/stestr_results.html.gz)
- [LP#1987092](https://logserver.rdoproject.org/56/44656/19/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/68cd2e4/logs/overcloud-controller-2/var/log/containers/stdouts/galera-bundle.log.txt.gz)
* ~~[`periodic-tripleo-ci-centos-9-bm_envC-3ctlr_1comp-featureset001-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-bm_envC-3ctlr_1comp-featureset001-master)~~ (not in criteria):
- [`cannot install the best update candidate for package gnutls-utils-3.7.3-10.el9.x86_64`](https://sf.hosted.upshift.rdu2.redhat.com/logs/42/44742/20/check-rdo/periodic-tripleo-ci-centos-9-bm_envC-3ctlr_1comp-featureset001-master/3df2a8f/job-output.txt)
* will be fixed by [chandan's revert](https://review.opendev.org/c/openstack/tripleo-quickstart/+/856582)
* ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm-2`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm-2)~~ (not in criteria):
- [node provisioning failed](https://logserver.rdoproject.org/88/44388/10/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-master-ibm-2/cd0c3bb/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz)
* [reruns 1](https://review.rdoproject.org/r/c/testproject/+/44656)
* [reruns 2](https://code.engineering.redhat.com/gerrit/c/testproject/+/426429) (internal)
* [reruns 3](https://review.rdoproject.org/r/c/testproject/+/44946)
* [reruns 4](https://review.rdoproject.org/r/c/testproject/+/44947)
* [reruns 5](https://review.rdoproject.org/r/c/testproject/+/44948)
* [reruns 6](https://review.rdoproject.org/r/c/testproject/+/44949)
- C9 Wallaby
+ last_promotion: ~~2022-09-09~~ 2022-09-13
+ [hash `f6749f9cda54d017021ab56bf4ec0958`](https://trunk.rdoproject.org/api-centos9-wallaby/api/civotes_agg_detail.html?ref_hash=f6749f9cda54d017021ab56bf4ec0958):
* yesterdays hash has passed today
+ [hash `ea8639e875ec8d7ee7158bce2cd3fa2b`](https://trunk.rdoproject.org/api-centos9-wallaby/api/civotes_agg_detail.html?ref_hash=ea8639e875ec8d7ee7158bce2cd3fa2b)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby):
- [TODO](https://logserver.rdoproject.org/55/36255/127/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/af08331)
* ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby)~~ (not in criteria):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-rbac-wallaby/ed83944)
* ~~[`periodic-tripleo-ci-centos-9-8-multinode-mixed-os`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-8-multinode-mixed-os)~~ (not in criteria):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-8-multinode-mixed-os/5d8e78e)
* [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby):
- [TODO](https://logserver.rdoproject.org/65/31165/89/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/3d50196)
* [reruns 1](https://review.rdoproject.org/r/c/testproject/+/44661)
- C8 Wallaby
+ last_promotion: 2022-09-11
+ [hash `21d2e33704a2f14d611cf4edff39f70e`](https://trunk.rdoproject.org/api-centos8-wallaby/api/civotes_agg_detail.html?ref_hash=21d2e33704a2f14d611cf4edff39f70e)
* ~~[`periodic-tripleo-ci-centos-8-skipped-standalone-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-skipped-standalone-wallaby)~~ (not in criteria):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-weekend/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-skipped-standalone-wallaby/a3ac814)
- C8 Train
+ last_promotion: 2022-09-12
+ [hash `54e3db5e5134970e993eeb7d124da84c`](https://trunk.rdoproject.org/api-centos8-train/api/civotes_agg_detail.html?ref_hash=54e3db5e5134970e993eeb7d124da84c)
* [`periodic-tripleo-ci-centos-8-standalone-full-tempest-api-train`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-full-tempest-api-train):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-api-train/65c3551)
* [`periodic-tripleo-centos-8-buildimage-overcloud-full-train`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-centos-8-buildimage-overcloud-full-train):
- [TODO](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-buildimage-overcloud-full-train/98d96fd)
* [reruns](https://review.rdoproject.org/r/c/testproject/+/44662)
- RHOS-17.1 on RHEL-9 **Refer to Bhagyashris' notes below**
- RHOS-17.1 on RHEL-8 **Refer to Bhagyashris' notes below**
- RHOS-17.0 on RHEL-9 **Refer to Bhagyashris' notes below**
- RHOS-16.2 on RHEL-8 **Refer to Bhagyashris' notes below**
* [C9 Master](http://dashboard-ci.tripleo.org/d/2tivP9BWz/component-pipeline?orgId=1):
- baremetal
+ last_promotion: 2022-09-12
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master):
* [tempest failures](https://logserver.rdoproject.org/openstack-component-baremetal/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master/913f5ca/logs/undercloud/var/log/tempest/stestr_results.html.gz)
* [other tempest failures](https://logserver.rdoproject.org/57/44657/13/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master/9a843f4/logs/undercloud/var/log/tempest/stestr_results.html.gz)
- network
+ last_promotion: 2022-09-08
+ ~~[`periodic-tripleo-ci-centos-9-undercloud-upgrade-network-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-undercloud-upgrade-network-master)~~ (not in criteria):
* [TODO](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-upgrade-network-master/9ac8b78)
+ [`periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master):
* older runs:
- 2022-09-12: [LP#1989197](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master/dc540aa/logs/undercloud/var/log/tempest/stestr_results.html.gz)
- 2022-09-12: [LP#1989197](https://logserver.rdoproject.org/57/44657/11/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master/b54f490/logs/undercloud/var/log/tempest/stestr_results.html.gz)
* [LP#1989197](https://logserver.rdoproject.org/57/44657/12/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master/7587e6a/logs/undercloud/var/log/tempest/stestr_results.html.gz)
* [LP#1989197](https://logserver.rdoproject.org/57/44657/13/check/periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-master/33472cf/logs/undercloud/var/log/tempest/stestr_results.html.gz)
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-master):
* [tempest failures](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-master/9b5506c)
* [overcloud deploy failed with `ERROR: Can't run container rabbitmq_bootstrap`](https://logserver.rdoproject.org/57/44657/13/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-master/36e896f/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
- security
+ last_promotion: 2022-09-11
+ ~~[`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master)~~:
* ~~[TODO](https://logserver.rdoproject.org/openstack-component-security/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-master/5cb9cdd)~~
- [reruns](https://review.rdoproject.org/r/c/testproject/+/44657)
* [C9 Wallaby](http://dashboard-ci.tripleo.org/d/UcMR8py7z/component-pipeline-wallaby-centos9?orgId=1):
- common
+ last_promotion: 2022-09-10
+ ~~[`periodic-tripleo-ci-centos-9-8-multinode-mixed-os-common-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-8-multinode-mixed-os-common-wallaby)~~ (not in criteria):
* [TODO](https://logserver.rdoproject.org/84/44684/7/check/periodic-tripleo-ci-centos-9-8-multinode-mixed-os-common-wallaby/5fa2da2)
- tripleo
+ last_promotion: 2022-09-12
+ [`periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby):
* [overcloud deploy failed with `ERROR: Can't run container container-puppet-ovn_controller`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/150abbf/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
- [reruns](https://review.rdoproject.org/r/c/testproject/+/44658)
* [C8 Wallaby](http://dashboard-ci.tripleo.org/d/ZLvFbT9Mz/component-pipeline-wallaby?orgId=1):
- all **GREEN** 😱
- ~~[reruns](https://review.rdoproject.org/r/c/testproject/+/44659)~~
* [C8 Train](http://dashboard-ci.tripleo.org/d/aJeQpzVGz/component-pipeline-train?orgId=1):
- network
+ last_promotion: 2022-09-09
+ ~~[`periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train)~~ (not in criteria):
* [TODO](https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-api-network-train/591113e)
- tripleo
+ last_promotion: 2022-09-10
+ [`periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train`](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train):
* [overcloud introspection failed](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train/41aaac6/logs/undercloud/home/zuul/overcloud_introspect.log.txt.gz)
- [reruns](https://review.rdoproject.org/r/c/testproject/+/44690)
* [RHOS-17.0 on RHEL-9](http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/lF7RUpsnk/rhel9-rhos17-full-component-pipeline?orgId=1):
* **Refer to Bhagyashris' notes below**
* [RHOS-16.2 on RHEL-8](http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/KyHCwLHMk/rhos-16-2-full-component-pipeline?orgId=1):
* **Refer to Bhagyashris' notes below**
### Downstream:
* rhos16.2 on rhel8:
* Promotion: 13-sept-2022
* blocked due to sc010 re-running jobs here: https://code.engineering.redhat.com/gerrit/c/testproject/+/422400/ **PASSED**
* rhos17.1 on rhel8
* Promotion: 12-sept-2022
* rhos17 on rhel9:
* Promotion: 11-Sept-2022
* failed fs001 job so re-ran here : https://code.engineering.redhat.com/gerrit/c/testproject/+/422400/10#message-60300b428a51d8e0c5639502640ead15c847b402 : **PASSED**
* rhos17.1 on rhel9:
* Promoted: 12-sept-2022
* blocked due full tempest api so re-ran here: https://code.engineering.redhat.com/gerrit/c/testproject/+/422400/10#message-60300b428a51d8e0c5639502640ead15c847b402 : **PASSED**
* jenkins job not triggerd:
* Note : Pinged atila: - After discussion with atila we got to know that currently we don't have trigger jobs for 17.1 on rhel9 and 8 so i am adding trigger job for 17.1 on 9 and 8 here: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/428003
* ~~fix for trigger jekins component rhso17.1 on rhel9 jobs is here: https://code.engineering.redhat.com/gerrit/c/rhos-qe-jenkins/+/428056 ~~
### Downstream component:
### Known Bugs
* https://bugs.launchpad.net/tripleo/+bug/1960355
- > Tempest tests test_oauth1_tokens.OAUTH1TokensTest are failing on tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-(master|wallaby)
- affecting OSP17.1 as well
* https://bugs.launchpad.net/tripleo/+bug/1987092 :warning: **BLOCKER** :warning:
- Pacemaker performance causes intermittent galera issues in loaded CI env
- jobs fail on `Check Keystone public endpoint status` or in other places
- [zuul/builds](https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-validation-master&skip=0)
- [affecting non-validation jobs such as periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby](https://bugs.launchpad.net/tripleo/+bug/1987092/comments/18)
* https://bugzilla.redhat.com/2105408 :warning: **BLOCKER** :warning:
- > RHOSP16.2 - Octavia tempest test "test_load_balancer_ipv4_CRUD" fails due to a kernel stacktrace (in a nested KVM setup)
- > reproducible only on RHEL 8.2 and RHEL 8.4 ... RHEL 8.6 is unaffected. So, please update the affected CI systems to RHEL 8.6.
- [latest status is tracked in cix card](https://trello.com/c/msIodUbD/2616-cixbz2105408osp162rhel8octavia-tempest-test-loadbalancerscenariotesttestloadbalanceripv4crud-failing-on-load-balancer-creation-t)
- blocking integration lines of c9 master (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master`) and c9 wallaby (`periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-wallaby`)
- [workaround (specifying `cpu_models` in libvirt)](
https://code.engineering.redhat.com/gerrit/c/openstack-tripleo-heat-templates/+/425806/) is
[currently being tested](https://code.engineering.redhat.com/gerrit/c/testproject/+/425432/)
* https://bugs.launchpad.net/tripleo/+bug/1988514
- [Trello](https://trello.com/c/TAL1jQcB/2692-cixlp1988514tripleociproa-source-image-rejected-none-of-the-signatures-were-accepted-reasons-open-etc-pki-rpm-gpg-rpm-gpg-key-re)
- On CIX board in lane `Waiting on external dependencies`
* https://bugs.launchpad.net/tripleo/+bug/1987323
- > FIPS + Manila + Ceph Quincy
- > ... known issue on the ceph side, ... already merged in master/quincy ...
> 17.2.4 should solve this issue, so we'll check during the next promotion.
- On CIX board in lane `Waiting on external dependencies`
- Not a blocker
* https://bugs.launchpad.net/tripleo/+bug/1988810
- Revert https://review.rdoproject.org/r/c/rdo-jobs/+/44788
* https://bugs.launchpad.net/tripleo/+bug/1989197
- > Tempest test `neutron_tempest_plugin.api.test_port_forwardings.PortForwardingTestJSON` fails on `periodic-tripleo-ci-centos-9-standalone-full-tempest-api-master`
* https://bugs.launchpad.net/tripleo/+bug/1989341
- > Tempest tests "tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON" etc. fail on {periodic-,}tripleo-ci-centos-8-9-multinode-mixed-os due to ovn controller/north version mismatch
* https://bugs.launchpad.net/tripleo/+bug/1989247
- > overcloud deploy fails with `sudo: unable to send audit message: Operation not permitted`
- > `Create containers managed by Podman for ... Failed containers: rabbitmq_bootstrap` (or others)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master`](https://logserver.rdoproject.org/56/44656/5/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/007c1da/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master`](https://logserver.rdoproject.org/57/44657/9/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-clients-master/3fb6e42/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* https://bugs.launchpad.net/tripleo/+bug/1989452
- > multiple periodic integration jobs fail configure-mirrors - Failed to connect to mirrors.centos.org port 443: No route to host
### Intermittent Failures
* https://bugs.launchpad.net/tripleo/+bug/1987616
- > cs9 ovb fs01 clients wallaby is failing with error - 'neutron.agent.dhcp.agent oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply'
- no CIX card
* https://bugs.launchpad.net/tripleo/+bug/1938987
- periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train
- > ERROR keystone.server.flask.request_processing.middleware.auth_context oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (1047, 'WSREP has not yet prepared node for application use')
- [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train`](https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/bd0a94d/logs/overcloud-controller-2/var/log/containers/keystone/keystone.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train`](https://logserver.rdoproject.org/62/44662/6/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/5753657/logs/overcloud-controller-0/var/log/containers/keystone/keystone.log.txt.gz)
* `TASK [freeipa-setup : Run the tripleo-ipa preparation script]`
- [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/205c81b/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/f940a55/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-wallaby/bfa6f03/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz)
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby`](https://logserver.rdoproject.org/40/44840/4/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/57331f2/job-output.txt)
* `Connection failure: The read operation timed out` in `TASK [Get current autoholds]`
- [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-wallaby/c3a98c9/job-output.txt)
* `Create containers managed by Podman for ... Failed containers: rabbitmq_bootstrap` (or others)
- > this has different root causes, watch out for log details!
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/61/44661/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/4926640/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
- [FAILURE in `tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby`](https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-tripleo-wallaby/df357c2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz)
* `FATAL | Wait for puppet host configuration to finish | undercloud | error`...
- `TASK [tripleo.operator.tripleo_undercloud_install : undercloud install]` in [job-output.txt](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/job-output.txt)
- `puppet-user: Error: Execution of '/bin/dnf -d 0 -e 1 -y install net-snmp' returned 1: Error: Error downloading packages:` in [logs/undercloud/home/zuul/undercloud_install.log.txt.gz](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz)
- [POST_FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby`](https://logserver.rdoproject.org/58/44658/2/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/54bb44c/logs/undercloud/home/zuul/undercloud_install.log.txt.gz)
* `Exception: Command '['ipmitool', ..., 'status']' timed out after 60 seconds"` during overcloud deploy
- [FAILURE in `periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby`](https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-wallaby/0a8588f/logs/undercloud/home/zuul/overcloud_node_provision.log.txt.gz)
### Fixed
---