owned this note changed 3 years ago
Published Linked with GitHub

Ruck Rover - 01 Apr 2022 - 07 Apr 2022

tags: ruck_rover
Previous RR notes: https://hackmd.io/mXb6zsGsTNWWO6Y6UpUNAg?view

Cockpit

Apr 7 2022

Apr 6 2022

Apr 05, 2022

Promotion blocker

It might be connected to: https://status.python.org/incidents/mxgkk3xxr9v7?u=v8pzlr5n28h8 Rechecking jobs to see if the issue is resolved (https://review.rdoproject.org/r/c/testproject/+/41370)

reference: https://lists.openstack.org/pipermail/openstack-discuss/2022-April/028037.html

The problem seems to be still present. I asked vexxhost for a confirmation wrt missing cache proxy.

FTR: The problem was connected to outdated pypi cache

05:57 <dpawlik> ProxyPass "/pypi/" "https://pypi.org/" ttl=120 keepalive=On retry=0 05:57 <dpawlik> ProxyPassReverse "/pypi/" "https://pypi.org/ 05:58 <dpawlik> and pypifiles 05:58 <bhjf> Title: PyPI · The Python Package Index (at pypi.org) 05:58 <dpawlik> CacheEnable disk "/pypifiles" 05:58 <dpawlik> ProxyPass "/pypifiles/" "https://files.pythonhosted.org/" ttl=120 keepalive=On retry=0 05:58 <dpawlik> ProxyPassReverse "/pypifiles/" "https://files.pythonhosted.org/"

06:18 <dpawlik> #status log flushink cache on afs mirror: htcacheclean -v -t -p /var/cache/httpd/ -l4096M 06:18 <sfstatusbot> dpawlik: finished logging 06:19 <dpawlik> #status log flushing cache on afs mirror: htcacheclean -v -t -p /var/cache/httpd/ -l4096M 06:19 <sfstatusbot> dpawlik: finished logging

Registry pruning issue: - https://review.rdoproject.org/r/c/rdo-infra/rdo-infra-playbooks/+/41574 CentOS-9-{master,wallaby} api endpoints were missing from the whitelist.

Other

Apr 04, 2022

11:49:45 amoralej | yoga is the last release of rdo on centos8

  • Downstream - lot sof stuck lines - need to be cleaned and see if downstream is working again. I cleared some line - pls get infra to clear the rest
  • Some long running lines in RDO zuul as well - may need clean up there
  • RDO - Pipelines were stuck cleard and may be need to reschedule those lines:
    • openstack-periodic-integration-stable1
    • openstack-periodic-integration-stable4
    • openstack-component-tempest
    • openstack-promote-component
  • Because of the zuul update lot's of jobs got failed with error: [Zuul] Log Stream did not terminate
  • PROMOTION BLOCKER: https://bugs.launchpad.net/tripleo/+bug/1967743

Upstream

Downstream failing components

  • periodic-tripleo-rhel-8-rhos-17-component-clients-promote-to-promoted-components

  • periodic-tripleo-rhel-8-rhos-17-component-network-promote-to-promoted-components

  • periodic-tripleo-rhel-8-rhos-17-component-tripleo-promote-to-promoted-components

  • periodic-tripleo-rhel-9-rhos-17-component-clients-promote-to-promoted-components

  • periodic-tripleo-rhel-8-rhos-16.2-component-network-promote-to-promoted-components

Apr 01, 2022

Mar 31, 2022

Prod Blocker

bz:2068055

cix:2069171 - downstream zuul still unavailable

CI Degraded

lp:1965540 revert created 835974, waiting for a merge of 835475. After it's merged, we can close cix:1965540

Select a repo