# Ruck Rover 2023-01-27 - 2023-02-02
[RDO Cockpit](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1) / [RHOS Cockpit](http://tripleo-cockpit.lab4.eng.bos.redhat.com)
[RDO Promoter](http://promoter.rdoproject.org/promoter_logs/) / [RHOS Promoter](http://10.0.110.143/promoter_logs/)

[OpenStack Program Meeting 2022](https://docs.engineering.redhat.com/pages/viewpage.action?spaceKey=PRODCHAIN&title=Meeting+notes)

Zuul Status:
* [opendev.org:openstack](https://zuul.opendev.org/t/openstack/status/)
* [rdoproject.org:rdoproject.org](https://review.rdoproject.org/zuul/status)
* [redhat.com:tripleo-ci-internal](https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status)

[Review List](/FGMoCiRfSNa8puA1BpTQ-Q)

###### tags: `ruck_rover`
###### Previous RR notes: https://hackmd.io/sacADiWwRO64_CzBHUFwiA

##### ruck & rover: pojadhav and dviroel

### NOTE: Ongoing bugs
:::info

Check/Gate blockers
* ~~https://bugs.launchpad.net/tripleo/+bug/2004557 flake8 is failing on tripleo-ci check pipeline jobs~~

Upstream blockers
* https://bugs.launchpad.net/tripleo/+bug/2004541 Hitting mirror issue everywhere
* https://bugs.launchpad.net/tripleo/+bug/2002478 centos9 wallaby scenario 10 fails octavia tempest test_traffic_ops

Downstream blockers:
* https://bugzilla.redhat.com/show_bug.cgi?id=2154179 periodic-tripleo-rhel-9-8-rhos-17.1-mixed-os-compute-internal failing consistently on task Run container image prepare for compute deploy
* https://bugzilla.redhat.com/show_bug.cgi?id=2166265 periodic-tripleo-ci-rhel-9-standalone-on-multinode-ipa-tripleo-rhos-17.1 consistently failing on task “install FreeIPA packages”
* https://bugzilla.redhat.com/show_bug.cgi?id=2166325 periodic-tripleo-ci-rhel-9-standalone-full-tempest-api-network-rhos-17 failing consistently with sporadic tempest tests failures
* https://bugzilla.redhat.com/show_bug.cgi?id=2166456 periodic-tripleo-build-containers-ubi-9-internal-rhel-9-build-push-upload-rhos-18 failing with “nothing provides network-scripts-openvswitch3.0”

Closed/Fixed:
* ~~https://bugs.launchpad.net/tripleo/+bug/2003865 mixed OS wallaby cs8/9 fails tempest network_basic_ops, security_groups~~
* https://bugs.launchpad.net/tripleo/+bug/2004141 ~~CS9 wallaby sc10-ovn-provider-standalone job failing consistently with error "libvirt.libvirtError: unsupported configuration: Emulator '/usr/libexec/qemu-kvm' does not support virt type 'kvm'"~~
* ~~https://bugs.launchpad.net/tripleo/+bug/2003882 unable to configure haproxy container with puppet (probably linked to an update in puppetlabs-haproxy)~~
* * ~~https://bugs.launchpad.net/tripleo/+bug/2000061 : CS-9 OVB fs001 tripleo master is unstable~~
* ~~https://bugs.launchpad.net/tripleo/+bug/2003658 playbooks/tripleo-rdo-base/container-login.yaml is failing on all RDO jobs~~
:::

## 2 Feb
### New bugs
* new issue seen all 17.1/17/upstream jobs (reported new bug https://bugs.launchpad.net/tripleo/+bug/2004541)
* https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-9-scenario002-standalone-common-rhos-17.1/0c61f84/job-output.txt - also upstream https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario000-multinode-oooq-container-updates-master/1b71b77/job-output.txt
* 2023-02-02 02:10:05.511877 | primary | ERROR: Could not install packages due to an OSError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))
* https://bugs.launchpad.net/tripleo/+bug/2004557

### Upstream lines
#### Promotions
- Master: 2023-02-01
- Zed: 2023-02-01
- Wallaby (cs9): 2023-02-01
- Wallaby (cs8): 2023-02-01
- Train: 2023-02-01 (no new content)

### Upstream job reckeck
- https://review.rdoproject.org/r/c/testproject/+/36719 (wallaby c8 & c9)

### Downstream lines
#### Promotions
- RHOS-18 RH9 - 2023-01-30
- RHOS-17.1 RH9 - 2023-01-30
- RHOS-17.1 RH8 - 2023-02-01 (no new content)
- RHOS-17 RH9 - 2023-01-30 (no new content)
- RHOS-16-2 RH8 - ~~2023-02-01~~ 2023-02-02

## 1 Feb
### New bugs
* https://bugzilla.redhat.com/show_bug.cgi?id=2166265 periodic-tripleo-ci-rhel-9-standalone-on-multinode-ipa-tripleo-rhos-17.1 consistently failing on task "install FreeIPA packages"
* https://bugzilla.redhat.com/show_bug.cgi?id=2166325 periodic-tripleo-ci-rhel-9-standalone-full-tempest-api-network-rhos-17 failing consistently with sporadic tempest tests failures
* last run passed, same hash https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-9-standalone-full-tempest-api-network-rhos-17&skip=0
* https://bugzilla.redhat.com/show_bug.cgi?id=2166456 periodic-tripleo-build-containers-ubi-9-internal-rhel-9-build-push-upload-rhos-18 failing with "nothing provides network-scripts-openvswitch3.0"
* **TODO**: need to ping *migarcia* since his was saying something about issues with rhosp-openvswitch updates (yesterday - 01/31)
* checking from morning - he is still away
* https://code.engineering.redhat.com/gerrit/c/testproject/+/437790 testing here as per shreshtha's lastest comment on CIX

### Upstream lines
#### Promotions \o/
- Master: ~~2023-01-31~~ 2023-02-01
- Zed: ~~2023-01-31~~ 2023-02-01
- Wallaby (cs9): ~~2023-01-30~~ 2023-02-01
- Wallaby (cs8): ~~2023-01-31~~ 2023-02-01
- Train: ~~2023-01-31~~ 2023-02-01

### Downstream lines
#### Promotions
- RHOS-18 RH9 - 2023-01-30
- RHOS-17.1 RH9 - 2023-01-30
- RHOS-17.1 RH8 - ~~2023-01-31~~ 2023-02-01
- RHOS-17 RH9 - 2023-01-30 (no new content)
- RHOS-16-2 RH8 - ~~2023-01-31~~ 2023-02-01

## 31 Jan
### New bugs

### Upstream lines
#### Promotions
- Master: ~~2023-01-30~~ 2023-01-31
- Zed: ~~2023-01-30~~ 2023-01-31
- Wallaby (cs9): 2023-01-30
- Wallaby (cs8): 2023-01-31
- Train: 2023-01-28
- (no new content https://trunk.rdoproject.org/centos8-train/tripleo-ci-testing/delorean.repo.md5 vs https://trunk.rdoproject.org/centos8-train/current-tripleo/delorean.repo.md5 )

### Upstream job reckeck
- https://review.rdoproject.org/r/c/testproject/+/36719 (master & zed)

### Downstream promotion
- RHOS-18 RH9 - 2023-01-30
- RHOS-17.1 RH9 - 2023-01-30
- RHOS-17.1 RH8 - ~~2023-01-28~~ 2023-01-31
- RHOS-17 RH9 - 2023-01-30 (no new content)
- RHOS-16-2 RH8 - ~~2023-01-29~~ 2023-01-31

### Downstream job reckeck
- https://code.engineering.redhat.com/gerrit/c/testproject/+/436232 (mixed_rhel job)

## 30 Jan
### New bugs
https://bugs.launchpad.net/bugs/2004141 CS9 wallaby sc10-ovn-provider-standalone job failing consistently with error "libvirt.libvirtError: unsupported configuration: Emulator '/usr/libexec/qemu-kvm' does not support virt type 'kvm'"

### Upstream lines
#### Promotions
- Master: ~~2023-01-29~~ 2023-01-30 (TODAY)
- Zed: ~~2023-01-28~~ 2023-01-30 (TODAY)
- Wallaby (cs9): ~~2023-01-24~~ 2023-01-30
- Wallaby (cs8): ~~2023-01-24~~ 2023-01-29
- Train: 2023-01-27 (no new hash)

### Upstream job reckeck
- https://review.rdoproject.org/r/c/testproject/+/36719

### Downstream promotion
- RHOS-18 RH9 - ~~2023-01-28~~ 2023-01-30
- RHOS-17.1 RH9 - ~~2023-01-28~~ 2023-01-30
- RHOS-17.1 RH8 - 2023-01-28 (no new hash)
- RHOS-17 RH9 - ~~2023-01-27~~ 2023-01-30
- RHOS-16-2 RH8 - 2023-01-29 (no new hash)

### Downstream job reckeck
- https://code.engineering.redhat.com/gerrit/c/testproject/+/437798

## 27 Jan
### Upstram lines
#### Promotions
- Master: 2023-01-27
- Zed: 2023-01-26
- Wallaby (cs9): 2023-01-24
- Wallaby (cs8): 2023-01-24
- Train: 2023-01-27

### Upstream job reckeck
- https://review.rdoproject.org/r/c/testproject/+/45162
- https://review.rdoproject.org/r/c/testproject/+/45217

### Downstream promotion
- RHOS-18 RH9 - 2023-01-26
- RHOS-17.1 RH9 - 2023-01-27
- running failing job here: https://code.engineering.redhat.com/gerrit/c/testproject/+/429803 [DNM] Test rhos17.1 on rhel9 jobs
- **jenkins jobs are GREEN** https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/pipeline/
- re-running failing job: https://code.engineering.redhat.com/gerrit/c/testproject/+/429803 [DNM] Test rhos17.1 on rhel9 jobs Job is GREEn
- **this hash "a30af6c027a874b67b27e43c72636a85" should promote in the next run of promoter**

- RHOS-17.1 RH8 - 2023-01-27
- https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/438865 Temp remove mixed rhel from criteria for BZ fix
- RHOS-17 RH9 - 2023-01-27
- running failing job here: https://code.engineering.redhat.com/gerrit/c/testproject/+/431169 [DNM] Test rhos17 on rhel9 failing job
- RHOS-16-2 RH8 - 2023-01-27