###### tags: `ruck_rover`
###### Previous RR notes: https://hackmd.io/x6p0UTBtSjGx3q9Dkxw2mg

##### ruck & rover: Adrian
##### ruck & rover alias: @ruckroverciframework

[CIX](https://issues.redhat.com/secure/RapidBoard.jspa?rapidView=17762)

[Next Gen cockpit](https://prometheus.monitoring.softwarefactory-project.io/grafana/d/5JC-ru3Vz/github-projects-workflow-status?orgId=1)
* [doc](https://docs.google.com/document/d/14CcHuMEACfXpAy1HkSc8s_CpN1KNC_POG2lzLnbKYLM/edit)

[RDO Cockpit](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1) / [RHOS Cockpit](http://tripleo-cockpit.lab4.eng.bos.redhat.com)
[RDO Promoter](http://promoter.rdoproject.org/promoter_logs/) / [RHOS Promoter](http://10.0.110.143/promoter_logs/)

[OpenStack Program Meeting indexes](https://docs.engineering.redhat.com/pages/viewpage.action?spaceKey=PRODCHAIN&title=Meeting+notes)

[Modernization CI](https://hackmd.io/f9JzEVjMQC-hkJu1L9eolw)

Zuul Status:
* [opendev.org:openstack](https://zuul.opendev.org/t/openstack/status/)
* [rdoproject.org:rdoproject.org](https://review.rdoproject.org/zuul/status)
* [redhat.com:tripleo-ci-internal](https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status)

Prow Status
* [Operators - Check/Gate failures](https://prow.ci.openshift.org/?job=pull-ci-openstack-k8s-operators*&state=failure)
* For now, we just look (and investigate) if a test has multiple failures across different PRs
* [Operators - Check/Gate status](https://prow.ci.openshift.org/?job=pull-ci-openstack-k8s-operators*)
* [pre-commit jobs](https://prow.ci.openshift.org/?job=pull-ci-openstack-k8s-operators*-main-precommit-check&state=failure)
* [pre-commit jobs status](https://prow.ci.openshift.org/?job=pull-ci-openstack-k8s-operators*-main-precommit-check)
* [unit tests jobs](https://prow.ci.openshift.org/?job=pull-ci-openstack-k8s-operators*-operator-main-unit&state=failure)
* [kuttl tests Jobs](https://prow.ci.openshift.org/?job=pull-ci-openstack-k8s-operators*-operator-build-deploy-kuttl&state=failure)
* [Tempest Jobs](https://prow.ci.openshift.org/?job=pull-ci-openstack-k8s-operators*-operator-build-deploy-tempest&state=failure)

[Review List](/FGMoCiRfSNa8puA1BpTQ-Q)

#### Podifed Container Build Lines
* [openstack-periodic-container-master-centos9](https://review.rdoproject.org/zuul/buildsets?pipeline=openstack-periodic-container-master-centos9)
* [openstack-periodic-container-antelope-centos9](https://review.rdoproject.org/zuul/buildsets?pipeline=openstack-periodic-container-antelope-centos9&skip=0)

---
:::danger

## Bugs

- https://issues.redhat.com/browse/OSPCIX-63 - tripleo-build-containers-rhel-[8|9] failing in "Execute build-report" task
- slow container upload
- Ticket opened with IT: INC2724809
- https://issues.redhat.com/browse/OSPCIX-64 - wallaby intermittent ovb failure - overcloud don't boot
- this issue is still under investigation with Harald's help. We are using this test project for debug: https://review.rdoproject.org/r/c/testproject/+/46960
- https://issues.redhat.com/browse/OSPCIX-46 - envB node provisioning issue on 17/9
- Possible hardware issue, IT ticket opened: RITM1579280
:::

:::info
## Closed bugs

- https://issues.redhat.com/browse/OSPCIX-70 - DLRN Api authentication error
:::

## Promotion Status

:::warning
#### Upstream
* Podified Master: 2023-09-17 (Green - no new hash)
* Podified Antelope: 2023-09-17 (Green - no new hash)
* CentOS 9 Wallaby : 2023-09-21 (Green - new hash)
* CentOS 8 Wallaby: 2023-09-21 (Green - new hash)
* CentOS 8 Train: 2023-09-19 (Green)

#### Downstream
* Podified RHEL 9 RHOS 18.0: 2023-09-18 (Green)
* RHEL 9 RHOS 17.1: 2023-09-18 (Yellow)
* RHEL 9 RHOS 17.0: 2023-09-01 (Green, no new hash)
* RHEL 8 RHOS 17.1: 2023-09-20 (Green)
* RHEL 8 RHOS 16.2: 2023-09-18 (Yellow)
---
:::

# Sep 21

*

# Sep 20

*

# Sep 19

* ~~oc kustomization task fail blocking podified promotion podified-multinode-edpm-deployment-crc job~~
* ~~https://github.com/openstack-k8s-operators/ci-framework/pull/570~~
* ~~https://review.rdoproject.org/zuul/build/6d456e7af26c4f1781e956cb356d7b7d~~
~~* https://issues.redhat.com/browse/OSPCIX-72~~
* PRs have been merged. Fixed and closed

# Sep 18
* ~~Check tox failure on collect-logs~~
* ~~https://redhat-internal.slack.com/archives/C03MD4LG22Z/p1694789838486839~~
* Fixed
* Some jobs are failing with NODE_FAILURE status:
* https://redhat-internal.slack.com/archives/C03MD4LG22Z/p1695020436754879?thread_ts=1695018600.920659&cid=C03MD4LG22Z
* https://review.rdoproject.org/zuul/builds?result=NODE_FAILURE&skip=0
* still hitting API errors AFAICT ... Time to open a ticket: https://support.vexxhost.com/hc/en-us/requests/364758
* CIX ticket opened in Jira: https://issues.redhat.com/browse/OSPCIX-71

# Sept 15
* Some jobs are failing with DLRN API authentication issue