Ruck Rover 2023-01-27 - 2023-02-02

RDO Cockpit / RHOS Cockpit RDO Promoter / RHOS Promoter

OpenStack Program Meeting 2022

Zuul Status:

• opendev.org:openstack
• rdoproject.org:rdoproject.org
• redhat.com:tripleo-ci-internal

Review List

tags: ruck_rover

Previous RR notes: https://hackmd.io/sacADiWwRO64_CzBHUFwiA

ruck & rover: pojadhav and dviroel

NOTE: Ongoing bugs

2 Feb

New bugs

• new issue seen all 17.1/17/upstream jobs (reported new bug https://bugs.launchpad.net/tripleo/+bug/2004541)
  • https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-9-scenario002-standalone-common-rhos-17.1/0c61f84/job-output.txt - also upstream https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario000-multinode-oooq-container-updates-master/1b71b77/job-output.txt
  • 2023-02-02 02:10:05.511877 | primary | ERROR: Could not install packages due to an OSError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))
• https://bugs.launchpad.net/tripleo/+bug/2004557

Upstream lines

Promotions

• Master: 2023-02-01
• Zed: 2023-02-01
• Wallaby (cs9): 2023-02-01
• Wallaby (cs8): 2023-02-01
• Train: 2023-02-01 (no new content)

Upstream job reckeck

• https://review.rdoproject.org/r/c/testproject/+/36719 (wallaby c8 & c9)

Downstream lines

Promotions

• RHOS-18 RH9 - 2023-01-30
• RHOS-17.1 RH9 - 2023-01-30
• RHOS-17.1 RH8 - 2023-02-01 (no new content)
• RHOS-17 RH9 - 2023-01-30 (no new content)
• RHOS-16-2 RH8 - 2023-02-01 2023-02-02

1 Feb

New bugs

• https://bugzilla.redhat.com/show_bug.cgi?id=2166265 periodic-tripleo-ci-rhel-9-standalone-on-multinode-ipa-tripleo-rhos-17.1 consistently failing on task "install FreeIPA packages"
• https://bugzilla.redhat.com/show_bug.cgi?id=2166325 periodic-tripleo-ci-rhel-9-standalone-full-tempest-api-network-rhos-17 failing consistently with sporadic tempest tests failures
  • last run passed, same hash https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-9-standalone-full-tempest-api-network-rhos-17&skip=0
• https://bugzilla.redhat.com/show_bug.cgi?id=2166456 periodic-tripleo-build-containers-ubi-9-internal-rhel-9-build-push-upload-rhos-18 failing with "nothing provides network-scripts-openvswitch3.0"
  • TODO: need to ping migarcia since his was saying something about issues with rhosp-openvswitch updates (yesterday - 01/31)
    • checking from morning - he is still away
    • https://code.engineering.redhat.com/gerrit/c/testproject/+/437790 testing here as per shreshtha's lastest comment on CIX

Upstream lines

Promotions \o/

• Master: 2023-01-31 2023-02-01
• Zed: 2023-01-31 2023-02-01
• Wallaby (cs9): 2023-01-30 2023-02-01
• Wallaby (cs8): 2023-01-31 2023-02-01
• Train: 2023-01-31 2023-02-01

Downstream lines

Promotions

• RHOS-18 RH9 - 2023-01-30
• RHOS-17.1 RH9 - 2023-01-30
• RHOS-17.1 RH8 - 2023-01-31 2023-02-01
• RHOS-17 RH9 - 2023-01-30 (no new content)
• RHOS-16-2 RH8 - 2023-01-31 2023-02-01

31 Jan

New bugs

Upstream lines

Promotions

• Master: 2023-01-30 2023-01-31
• Zed: 2023-01-30 2023-01-31
• Wallaby (cs9): 2023-01-30
• Wallaby (cs8): 2023-01-31
• Train: 2023-01-28
  • (no new content https://trunk.rdoproject.org/centos8-train/tripleo-ci-testing/delorean.repo.md5 vs https://trunk.rdoproject.org/centos8-train/current-tripleo/delorean.repo.md5 )

Upstream job reckeck

• https://review.rdoproject.org/r/c/testproject/+/36719 (master & zed)

Downstream promotion

• RHOS-18 RH9 - 2023-01-30
• RHOS-17.1 RH9 - 2023-01-30
• RHOS-17.1 RH8 - 2023-01-28 2023-01-31
• RHOS-17 RH9 - 2023-01-30 (no new content)
• RHOS-16-2 RH8 - 2023-01-29 2023-01-31

Downstream job reckeck

• https://code.engineering.redhat.com/gerrit/c/testproject/+/436232 (mixed_rhel job)

30 Jan

New bugs

https://bugs.launchpad.net/bugs/2004141 CS9 wallaby sc10-ovn-provider-standalone job failing consistently with error "libvirt.libvirtError: unsupported configuration: Emulator '/usr/libexec/qemu-kvm' does not support virt type 'kvm'"

Upstream lines

Promotions

• Master: 2023-01-29 2023-01-30 (TODAY)
• Zed: 2023-01-28 2023-01-30 (TODAY)
• Wallaby (cs9): 2023-01-24 2023-01-30
• Wallaby (cs8): 2023-01-24 2023-01-29
• Train: 2023-01-27 (no new hash)

Upstream job reckeck

• https://review.rdoproject.org/r/c/testproject/+/36719

Downstream promotion

• RHOS-18 RH9 - 2023-01-28 2023-01-30
• RHOS-17.1 RH9 - 2023-01-28 2023-01-30
• RHOS-17.1 RH8 - 2023-01-28 (no new hash)
• RHOS-17 RH9 - 2023-01-27 2023-01-30
• RHOS-16-2 RH8 - 2023-01-29 (no new hash)

Downstream job reckeck

• https://code.engineering.redhat.com/gerrit/c/testproject/+/437798

27 Jan

Upstram lines

Promotions

• Master: 2023-01-27
• Zed: 2023-01-26
• Wallaby (cs9): 2023-01-24
• Wallaby (cs8): 2023-01-24
• Train: 2023-01-27

Upstream job reckeck

• https://review.rdoproject.org/r/c/testproject/+/45162
• https://review.rdoproject.org/r/c/testproject/+/45217

Downstream promotion

• RHOS-18 RH9 - 2023-01-26

• RHOS-17.1 RH9 - 2023-01-27

  • running failing job here: https://code.engineering.redhat.com/gerrit/c/testproject/+/429803 [DNM] Test rhos17.1 on rhel9 jobs
  • jenkins jobs are GREEN https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/pipeline/
  • re-running failing job: https://code.engineering.redhat.com/gerrit/c/testproject/+/429803 [DNM] Test rhos17.1 on rhel9 jobs Job is GREEn
  • this hash "a30af6c027a874b67b27e43c72636a85" should promote in the next run of promoter

• RHOS-17.1 RH8 - 2023-01-27

  • https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/438865 Temp remove mixed rhel from criteria for BZ fix

• RHOS-17 RH9 - 2023-01-27

  • running failing job here: https://code.engineering.redhat.com/gerrit/c/testproject/+/431169 [DNM] Test rhos17 on rhel9 failing job

• RHOS-16-2 RH8 - 2023-01-27