---
# System prepended metadata

title: Ruck Rover 2022-11-25 - 2022-12-01
tags: [ruck_rover]

---

# Ruck Rover 2022-11-25 - 2022-12-01
[RDO Cockpit](http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1) / [RHOS Cockpit](http://tripleo-cockpit.lab4.eng.bos.redhat.com)
[RDO Promoter](http://promoter.rdoproject.org/promoter_logs/) / [RHOS Promoter](http://10.0.110.143/promoter_logs/)

[OpenStack Program Meeting 2022](https://docs.engineering.redhat.com/pages/viewpage.action?spaceKey=PRODCHAIN&title=Meeting+notes)


Zuul Status:
* [opendev.org:openstack](https://zuul.opendev.org/t/openstack/status/)
* [rdoproject.org:rdoproject.org](https://review.rdoproject.org/zuul/status)
* [redhat.com:tripleo-ci-internal](https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status)

[Review List](/FGMoCiRfSNa8puA1BpTQ-Q)

###### tags: `ruck_rover`
###### Previous RR notes: https://hackmd.io/Vn_zjslARMG5Crs4YwnSqw

##### ruck & rover: pooja & ananya

:::info


### Active Bugs
* ~~https://bugs.launchpad.net/tripleo/+bug/1997939~~ (c8 integration)
* ~~https://bugs.launchpad.net/tripleo/+bug/1998098~~ (validations)
* https://bugs.launchpad.net/tripleo/+bug/1998211 (fs64 zed overcloud deploy)
* https://bugs.launchpad.net/tripleo/+bug/1998227 (tempest c9 fs001)
* https://bugs.launchpad.net/tripleo/+bug/1998348 (selinux) (**TODO** close eod)
* https://bugzilla.redhat.com/show_bug.cgi?id=2148354 (rhos17/17.1 fs35)
* https://bugzilla.redhat.com/show_bug.cgi?id=2149963 (jenkins jobs 17.1 on rhel9)

:::
## Dec 01
### Gate

No particular job is red but I see some Retry limits (**TODO** monitor)

### New bugs
https://bugzilla.redhat.com/show_bug.cgi?id=2149963
    
### Upstream promotion pipelines
* Master: 2022-11-28
    * https://bugs.launchpad.net/tripleo/+bug/1998227 
        * tp: https://review.rdoproject.org/r/c/testproject/+/46302
* Zed: 2022-11-25
    * fs64: 
        * https://bugs.launchpad.net/tripleo/+bug/1998211
        * tp: https://review.rdoproject.org/r/c/testproject/+/36254
* Wallaby c9: 30 nov
    * tp: **TODO**
* Wallaby c8: 30 nov
    * tp: **TODO**
* Train c8: 30 nov
    * tp: **TODO**
### Upstream component pipelines
* c9: 
    * fs001s : https://bugs.launchpad.net/tripleo/+bug/1998227
    * standalones: https://bugs.launchpad.net/tripleo/+bug/1998348 (this cleared on its own, will close by eod)
* c8:
    * tp: **TODO**
### Downstream promotion pipelines
- 17/9 - ~~30th Nov (Promoted yesterday)~~ 01st Dec (promoted today)
- 17.1/9 - 29th Nov (Blocked on jenkins jobs)
    - chasing promotions https://code.engineering.redhat.com/gerrit/c/testproject/+/209874/55
    - reported a bug for jenkins job failure https://bugzilla.redhat.com/show_bug.cgi?id=2149963
- 17.1/8 - 30th Nov (Promoted yesterday)
- 16.2/8 - 30th Nov (Promoted yesterday) No new hash

### Downstream component pipelines
- 17/9 - All green except network component
    - Rerun https://code.engineering.redhat.com/gerrit/c/testproject/+/436232/5
- 17.1/9 - All green except network component
    - Rerun https://code.engineering.redhat.com/gerrit/c/testproject/+/436232/5
- 17.1/8 - No jobs in criteria
- 16.2/8 - All green

## Nov 30
### Gate
    No blockers
### New bugs
### Upstream promotion pipelines
* Master: 2022-11-28
    * https://bugs.launchpad.net/tripleo/+bug/1998227 
        * tp: https://review.rdoproject.org/r/c/testproject/+/46302
* Zed: 2022-11-25
    * fs64: 
        * https://bugs.launchpad.net/tripleo/+bug/1998211
* Wallaby c9: ~~2022-11-28~~ PROMOTED TODAY!!
    * https://review.rdoproject.org/r/c/testproject/+/46256
* c8 ~~(wallaby 2022-11-23) (train 2022-11-24)~~ : PROMOTED TODAY!! https://review.rdoproject.org/r/c/testproject/+/46301

### Upstream component pipelines
* Master, Zed:
    * https://bugs.launchpad.net/tripleo/+bug/1998348
    * https://bugs.launchpad.net/tripleo/+bug/1998227 
* Wallaby c9
    * https://bugs.launchpad.net/tripleo/+bug/1998348

### Downstream promotion pipelines
- 17/9 - 30th Nov
    - rerunning sc12 standalone https://code.engineering.redhat.com/gerrit/c/testproject/+/209874/54
- 17.1/9 - 29th Nov
- 17.1/8 - 29th Nov
- 16.2/8 - 28th Nov
    - rerunning tempest api https://code.engineering.redhat.com/gerrit/c/testproject/+/209874/54

### Downstream component pipelines
- 17/9 - 
    - Security and network component 
    - Rerunning https://code.engineering.redhat.com/gerrit/c/testproject/+/436232/3
- 17.1/9 - 
    - network component
    - Rerunning https://code.engineering.redhat.com/gerrit/c/testproject/+/436232/3
- 17.1/8 - No jobs in criteria
- 16.2/8 - All green


## Nov 29
### Gate
### New bugs
#### Upstream promotion pipelines
* Master:
    * https://bugs.launchpad.net/tripleo/+bug/1998227 

* Zed:
    * https://review.rdoproject.org/r/c/testproject/+/46287
    * fs64: 
        * https://bugs.launchpad.net/tripleo/+bug/1998211

* Wallaby c9:
    * https://review.rdoproject.org/r/c/testproject/+/46256
 
#### Upstream Component pipelines

* https://review.rdoproject.org/r/c/testproject/+/46292
* https://review.rdoproject.org/r/c/testproject/+/46293
- Master
    - validation 
        - https://review.rdoproject.org/r/c/testproject/+/46282
        - https://review.rdoproject.org/r/c/testproject/+/33582
- Zed 
    - validation 
        - https://review.rdoproject.org/r/c/testproject/+/46283
    - security
        - fs39 failed with random failures
    - network
        - fs01 failed with random failures.
- Wallaby C9
    - common 
        - standalone blocked on https://bugs.launchpad.net/tripleo/+bug/1997939
- Wallaby C8
    - tripleo 
        - standalone job blocked on https://bugs.launchpad.net/tripleo/+bug/1997939
    - tempest
        - standalone job blocked https://bugs.launchpad.net/tripleo/+bug/1997939
    - security
        - standalone job blocked https://bugs.launchpad.net/tripleo/+bug/1997939
    - network
        - standalone job blocked https://bugs.launchpad.net/tripleo/+bug/1997939
    - https://review.rdoproject.org/r/c/testproject/+/40897
- Train C8 
    - tripleo 
        - fs01 blocked on https://bugs.launchpad.net/tripleo/+bug/1997939
    - common 
        - sc04 standalone blocked on https://bugs.launchpad.net/tripleo/+bug/1997939


#### Downstream promotion pipelines
- 17/9 - 28th Nov  Line currently running
- 17.1/9 - 28th Nov fs35 and sc01 standalone failed
    - Rerun : https://code.engineering.redhat.com/gerrit/c/testproject/+/209874/52
- 17.1/8 - 28th Nov All green
- 16.2/8 - 28th Nov All green 
#### Downstream component promotion pipelines
- 17/9 - All green except sc04 common component job 
    - Rerun : https://code.engineering.redhat.com/gerrit/c/testproject/+/436232/2
- 17.1/9 - All green except tempest api network component job
    - Rerun : https://code.engineering.redhat.com/gerrit/c/testproject/+/436232/2
- 17.1/8 - No job in criteria
- 16.2/8 - All green 


## Nov 28

### Gate:
### New bugs: 
* https://bugs.launchpad.net/tripleo/+bug/1997939 (Affects wallaby and train c8)
    * tp: https://review.rdoproject.org/r/c/testproject/+/46272
#### Upstream promotion pipelines
* Master: 28th
    - https://review.rdoproject.org/r/c/testproject/+/46253
* Zed: 25th:
    * https://review.rdoproject.org/r/c/testproject/+/46242
    * featureset064:
        * Could not create '/home/zuul/overcloud-hardened-uefi-full.raw': Permission denied
        * Overcloud dceploy failed
* Wallaby c9: 27th
    * https://review.rdoproject.org/r/c/testproject/+/46256
* Wallaby c8:  RED
    * https://bugs.launchpad.net/tripleo/+bug/1997939 
* Train: RED
    * https://bugs.launchpad.net/tripleo/+bug/1997939
#### Upstream Component pipelines
* Master
    * tripleo (tempest tests failed)
        * ~~https://review.rdoproject.org/r/c/testproject/+/46257~~
    * validation
        * https://bugs.launchpad.net/tripleo/+bug/1998098
* Zed
    * periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-zed 
        * deploy-steps-tasks-step-0.yaml times out (**TODO** bug)
        * https://logserver.rdoproject.org/openstack-component-security/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-security-zed/0953c50/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz
* Wallaby c9:
    * https://review.rdoproject.org/r/c/testproject/+/46269
* Walllaby c8:
    * tempest, tripleo, network - standalone job failing with https://bugs.launchpad.net/tripleo/+bug/1997939
* Train c8:
    * common - 
        * standalone - https://bugs.launchpad.net/tripleo/+bug/1997939
    * tripleo
        * fs001 - https://bugs.launchpad.net/tripleo/+bug/1997939
        * sc04 standalone - https://bugs.launchpad.net/tripleo/+bug/1997939
#### Downstream promotion pipelines
- 17/9 - 28th Nov fs35 by skipping https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/865797
- 17.1/9 - 28th Nov fs35 by skipping https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/865797
- 17.1/8 - 26th Nov 
- 16.2/8 - 28th Nov(Today) - ~~tempest-scenario and sc01 standalone failing with randaom tempest test.~~
    - Rerun (got sucess) https://code.engineering.redhat.com/gerrit/c/testproject/+/209874/50

#### Downstream component promotion pipelines
- 17/9 - All green except tripleo component jenkins job
- 17.1/9 - All green
- 17.1/8 - No job in criteria
- 16.2/8 - All green exceot tripleo component jenkins job


## Nov 25

### Gate: 
Rechecked on https://review.opendev.org/c/openstack/heat/+/802139/ (failed once)

### New bugs: 
* https://bugzilla.redhat.com/show_bug.cgi?id=2148354 periodic-tripleo-ci-rhel-9-ovb-3ctlr_1comp-featureset035-internal-rhos-17.1 failing consistently with set of keystone_tempest_plugin tests with error "Unexpected content type provided"
* https://bugs.launchpad.net/tripleo/+bug/1997939 tripleo-ci-centos-8-standalone in stable/wallaby consistently fails because of the failure during creating galera resource


#### Upstream promotion pipelines
- Master - 
    -  https://review.rdoproject.org/r/c/testproject/+/46241 
- Zed - promoted
    - ~~https://review.rdoproject.org/r/c/testproject/+/46242~~
- C9 Wallaby - 
    - running

- C8 wallaby - 
    - https://review.rdoproject.org/r/c/testproject/+/46243 (rerunning the olds registry push job till Amol's patches merge)
 
- C8 Train -
 
#### Upstream Component pipelines
- Master
    - clients
        - fs001 ovb (random tempest tests failing)
    - network
        - fs001 ovb (random tempest tests failing)
    - Reruns (https://review.rdoproject.org/r/c/testproject/+/33582)

- Zed 
    - Reruns (https://review.rdoproject.org/r/c/testproject/+/36719)
- C9 wallaby
    - tempest - random tests failing 
- C8 wallaby - 
    - Reruns (https://review.rdoproject.org/r/c/testproject/+/40897)
- C8 train - 
    - Reruns (https://review.rdoproject.org/r/c/testproject/+/36478)

#### Downstream promotion pipelines
- 17/9 - 22nd Nov (Line running)
- 17.1/9 - 22nd Nov - fs35 failing with tempest tests
- 17.1/8 - 22nd Nov (All green)
- 16.2/8 - 23rd Nov
    - fs35 failed once  [Rerun]https://code.engineering.redhat.com/gerrit/c/testproject/+/209874
(got a green run https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/ed489e014c584d8d833e9d070132ed6a)

#### Downstream component promotion pipelines
- 17/9 - All green
- 17.1/9 - network and tripleo [Rerun] https://code.engineering.redhat.com/gerrit/c/testproject/+/436232
(got a green run https://code.engineering.redhat.com/gerrit/c/testproject/+/436232/1#message-c943b28d5be1c620a8053a5ef8142341f5ea8c40)
- 17.1/8 - No job in criteria
- 16.2/8 - All green
