Ruck and Rover notes #32

tags: ruck_rover

• Ruck and Rover notes #32
  • on-going issues
    • Reviews / Fixes
    • Bugs reported
  • 09/09/2020
  • 09/08/2020
  • 09/07/2020
  • 09/04/2020
  • 09/03/2020
  • 09/02/2020
  • 09/01/2020
  • 08/31/2020
    • Tripleo
  • 08/28/2020
    • Tripleo
  • 08/27/2020
    • Tripleo
    • Others
  • 08/26/2020
    • Tripleo
    • Other
  • 08/25/2020
    • TripleO
    • Other
  • 08/24/2020
    • TripleO
    • Others
    • OSP
• 08/21/2020
  • • tripleo
    • osp
  • Bugs from sprint 31
    • Patches in check/gate
    • Bugs still seen
    • Bugs no activity yet
    • Keep an eye on

on-going issues

Reviews / Fixes

Bugs reported

09/09/2020

• triplo-ci-centos-7-standalone-upgrade-train job is failing LP: 1895005
• https://bugs.launchpad.net/tripleo/+bug/1894953 - tempest failure, seems to be nova bug
  • https://review.opendev.org/#/c/750586/ Added on skiplist

09/08/2020

• Python tripleoclient -tox-py27 stable/train broken 1894841

09/07/2020

• https://bugs.launchpad.net/tripleo/+bug/1894664 - Train containers job failing
• CentOS 8 train jobs are failing. Log URL

  ​​​​2020-09-04 07:24:26 | Host 192.168.24.16 not found in /home/zuul/.ssh/known_hosts
  ​​​​2020-09-04 07:24:26 | Overcloud configuration failed.
  

09/04/2020

• Master Promotion
  • Extra openstack- prefix.. caused master promotion failure.
  • http://paste.openstack.org/show/797455/
  • AI: Wes

09/03/2020

• Upstream is red, https://bugs.launchpad.net/tripleo/+bug/1893339
  • https://review.opendev.org/#/c/748609/
  • Error:

  ​​​​error={"changed": false, "error": "Not found image: https://registry-1.docker.io/v2/tripleomaster/centos-binary-swift-object/manifests/df855c291ed5531d780dbba4a768e1f3", "msg": "Error running container image prepare: Not found image: https://registry-1.docker.io/v2/tripleomaster/centos-binary-swift-object/manifests/df855c291ed5531d780dbba4a768e1f3", "params": {}, "success": false}
  

09/02/2020

• Nothing news so far

09/01/2020

• Same errors still happening

08/31/2020

• Same errors as last week

Tripleo

• Promotion Blocker
  • https://bugs.launchpad.net/tripleo/+bug/1893618 - Tempest test_shelve_unshelve_server failing constantly
  • https://bugs.launchpad.net/tripleo/+bug/1893621 - Could not establish a connection to the Zaqar websocket
    • https://review.opendev.org/#/c/748673/
  • Ussuri had condition race in podman, Sagi already backport his patch
    • https://review.opendev.org/#/q/72f0e09019ee13ad306287a8f60faba30d4fb60e
  • Rocky and Stein tempest container issue
    • https://review.opendev.org/#/c/748955/ - Thanks to Yatin

08/28/2020

Tripleo

• Promotion Blocker:
  • https://bugs.launchpad.net/tripleo/+bug/1893339
    • https://review.opendev.org/#/c/748974/ - Added test on skiplist

08/27/2020

Tripleo

• https://bugs.launchpad.net/tripleo/+bug/1893173 -
  Error while evaluating a Resource Statement

  • https://review.opendev.org/#/c/748360/

• https://bugs.launchpad.net/tripleo/+bug/1893192 - octavia_tempest_plugin.tests.scenario.v2.test_listener.ListenerScenarioTest failed in scen010 ovn

Others

08/26/2020

Same issues from yesterday happening again, that's the reason there's nothing here today

Tripleo

• Patches to review:
  • https://review.opendev.org/748160

Other

08/25/2020

TripleO

• Timeout on periodic-tripleo-ci-centos-8-scenario010-standalone-master

  • Happened only today, should a bug be oppened? (check with wes)

• Error in tempest-api job https://bugs.launchpad.net/tripleo/+bug/1892837

  • https://review.opendev.org/#/c/747868/ Added test on skiplist

Other

08/24/2020

TripleO

• Centos8-scenario012 failing on stable/train: Tempest failure

  • Test is added in skiplist for master, train and usuri
  • LP

• Periodic Scenario 010 ovn provider job failing with ovn provider is not enabled https://bugs.launchpad.net/tripleo/+bug/1892682

  • Job tripleo-ci-centos-7-scenario009-multinode-oooq-container continuously failing.

• Race condition in podman https://bugs.launchpad.net/tripleo/+bug/1892701

  • https://review.opendev.org/747685 might help to avoid race condition

• Upgrade jobs failing in create container https://bugs.launchpad.net/tripleo/+bug/1892702

Others

• Pre-commit bug https://github.com/pre-commit/pre-commit/issues/1575

  • Did not fix the problem, opening another one

• https://github.com/pre-commit/pre-commit/issues/1578

  • https://review.opendev.org/#/c/747649/ pin pre-commit to < 2.7.0

• Jobs getting timed out: This issue is seen in few jobs.

  • ERROR: RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/tripleo-ci/playbooks/tripleo-ci/run-v3.yaml@master]
  • https://643169a2915789dbd696-0cff35b3d101f5c13616cb96b9f1415f.ssl.cf5.rackcdn.com/746219/6/gate/tripleo-ci-centos-8-undercloud-containers/48539e2/job-output.txt
  • periodic-tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-ussuri
  • tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates

OSP

08/21/2020

tripleo

handoff -> wes

osp

Bugs from sprint 31

Patches in check/gate

• https://bugs.launchpad.net/tripleo/+bug/1892008 -
  periodic OVB train centos 7 failing overcloud configuration (config download) Invalid cross-device link

• https://bugs.launchpad.net/tripleo/+bug/1891372 -
  rocky periodic jobs are failing with " Error: image tripleorocky/centos-binary-tempest:9801dc7461cbd6cbd73868e72e74d21d586c6708_fbb4de96-updated-20200812131025 not found"

Bugs still seen

• https://bugs.launchpad.net/tripleo/+bug/1892326 -
  Jobs failing with RETRY_LIMIT with primary | /bin/sh: line 1: git: command not found at prepare-workspace-git : Clone cached repo to workspace

Bugs no activity yet

• https://bugs.launchpad.net/tripleo/+bug/1892169 -
  periodic centos 8 train FS20 tempest fails test_server_connectivity_resize (in fs020 not part of train promotion)

Keep an eye on

• https://review.opendev.org/#/c/747129