Scenario 1 (Green - Merged):-

Patch - code.engineering.redhat.com/gerrit/#/c/200373

Testproject Patch with run - https://code.engineering.redhat.com/gerrit/#/c/199982/

Scenario 4 (Green - Merged):-

Patch - code.engineering.redhat.com/gerrit/#/c/200505

Testproject with job run - code.engineering.redhat.com/gerrit/#/c/200636

Scenario 2 (Green - Merged)

Patch - https://code.engineering.redhat.com/gerrit/#/c/200511/

Testproject with job run - https://code.engineering.redhat.com/gerrit/#/c/200637/

Scenario10 (Green - Merged)

Patch - code.engineering.redhat.com/gerrit/#/c/200673

Testproject with job run - code.engineering.redhat.com/gerrit/#/c/200674

Fixed after moving from validate-tempest to os_tempest

Scenario7 (Green - Merged)

Patch - https://code.engineering.redhat.com/gerrit/#/c/201430/

Testproject with job run - https://code.engineering.redhat.com/gerrit/#/c/200637/

Track of issue faced earlier - rabbitmq can't start properly in some sceanrios when ipv6 is entirely disabled on the system. ipv6 was getting disabled due to downstream nodepool specific customization.

After fix - https://code.engineering.redhat.com/gerrit/#/c/201431/1 and the new image builds - https://sf.hosted.upshift.rdu2.redhat.com/nodepool-log/cloud-rhel-8-1-0000013385.log - we are good

Multinode featureset010 (Red)

Patch - https://code.engineering.redhat.com/gerrit/#/c/201376/

testproject - https://code.engineering.redhat.com/gerrit/#/c/201382/

Failing with emit_releases_file.py: error: argument stable-release: invalid choice: 'rhos-17

https://sf.hosted.upshift.rdu2.redhat.com/logs/82/201382/1/check/periodic-tripleo-ci-rhel-8-multinode-1ctlr-featureset010-rhos-17/eff9e9e/job-output.txt

2020-05-26 06:33:35.825941 | primary | usage: emit_releases_file.py [-h] --stable-release
2020-05-26 06:33:35.826019 | primary |                              {newton,ocata,pike,queens,rocky,stein,train,ussuri,master}
2020-05-26 06:33:35.826028 | primary |                              --distro-name {centos} --distro-version {7,8}
2020-05-26 06:33:35.826035 | primary |                              --featureset-file FEATURESET_FILE
2020-05-26 06:33:35.826040 | primary |                              [--output-file OUTPUT_FILE] [--log-file LOG_FILE]
2020-05-26 06:33:35.826045 | primary |                              [--upgrade-from] [--is-periodic]
2020-05-26 06:33:35.826061 | primary | emit_releases_file.py: error: argument --stable-release: invalid choice: 'rhos-17' (choose from 'newton', 'ocata', 'pike', 'queens', 'rocky', 'stein', 'train', 'ussuri', 'master')

Extra Discussion

Ceph container parameters

It looks like the internal jobs were not looking for the internal ceph containers in the correct locations.
The review above try to address that.
NOTE: tripleo-ci-internal tenant went down on Monady 05/18.
This was CIX'ed. If you catch infra, you can rerun and check.

Note also the reparenting from the job that would be in the integration line (to match the upstream RDO pattern).

@rlandy Thank you for reparenting.

* Marios also pointed considering using periodic-tripleo-ci-rhel-8-standalone-component-rhos-17 as parent to get osp-17 release vars 

* https://code.engineering.redhat.com/gerrit/#/c/200373/2/zuul.d/component-jobs.yaml
* https://code.engineering.redhat.com/gerrit/#/c/200373/5/zuul.d/component-jobs.yaml

 and same thought hit me as well but at that time i thought the approach I used earlier was slight better -  Thinking about future releases  when rhos18/19 will comes, we can just reuse "periodic-tripleo-ci-rhel-8-scenario001-standalone-component "(if we keep it generic ) as a parent job to create 

- periodic-tripleo-ci-rhel-8-scenario001-standalone-component-rhos-17 ( passing release/branch related vars there)
- periodic-tripleo-ci-rhel-8-scenario001-standalone-component-rhos-18 
- periodic-tripleo-ci-rhel-8-scenario001-standalone-component-rhos-19 

But I am okay with your approach if this keep things consistent with RDO.

My scenario1 Patch need slight changes.. I made them and rechecked your testpatch.

Rework parenting??

@rlandy @marios fyi.. Gathered some thoughts here on possible parenting design.. Will need your suggestions to help in opting the best possible solution here. (We are still currently focusing on sceanrio001 as rest other sceanrios will follow the pattern - scenario001 currently is running good but failing due to prometheues containers)

Context:-

In rdo/upstream whenever we create a new scenario job - we create it for master and whenever there is branching(new release) - we inherit from master for the new branched job(for ex: ussuri).

example:-

But in downstream there is no Master to follow , Right now in downstream(In our initial work[1]) we are treating rhos-17 as master. If we follow same pattern when RHOSP18 will come - in order to not repeat ourselves we can possibily use rhos-17 as parent and override release specific vars(This can just work fine) - but this can create a challenge if in future we decided to drop rhos17 jobs.

[1] https://code.engineering.redhat.com/gerrit/#/c/200373/14/zuul.d/standalone-jobs.yaml

What can be done -

First possible solution:-

I am wondering if in downstream we should first just create generic jobs(without any release specific vars) instead of treating 17 as master, This way we can just reuse the generic jobs to create rhos17/18/19.. jobs(as they come). We will diverge from RDO/Upstream pattern but it seems needed as we don't have a master to follow.

I brainstormed to rework https://code.engineering.redhat.com/gerrit/#/c/200373/14/zuul.d/standalone-jobs.yaml to something like below:-

standalone-jobs.yaml

  • Add an anchor on rhos17 vars(Line 185) of existing job periodic-tripleo-ci-rhel-8-standalone-rhos-17
 175 # openstack-periodic-rhos-17 jobs
 176 - job:
 177     name: periodic-tripleo-ci-rhel-8-standalone-rhos-17
 178     parent: tripleo-ci-rhel-8-standalone-internal-base-dlrn
 179     # This need to change to branch used for integration
 180     branches: .*
 181     voting: false
 182     pre-run:
 183       - playbooks/configure-rhel.yaml
 184       - playbooks/pre-rhos.yaml
 185     vars: &vars_rhos17  ----> Add an anchor here
 186       dlrnapi_host: "osp-trunk.hosted.upshift.rdu2.redhat.com"
 187       dlrnapi_protocol: "http"
 188       release: master
 189       osp_release: "rhos-17"
 190       branch_override: "stable/rhos-17"
 191       rhos_release_version: 17
 192       release_file: promotion-testing-hash-rhos-17
 193       promote_source: tripleo-ci-testing
 194       featureset_override:
 195         tempest_tempest_conf_overrides:
 196           image.image_path: 'http://images.rdoproject.org/cirros/cirros-0.4.0-x86_64-disk.img'

- job:
    name: periodic-tripleo-ci-rhel-8-scenario001-standalone
+-  parent: tripleo-ci-rhel-8-standalone-internal-base-dlrn  --> ( changed from  periodic-tripleo-ci-rhel-8-standalone-rhos-17)
+   branches: .*    
+   voting: false
+   pre-run:
+     - playbooks/configure-rhel.yaml
+     - playbooks/pre-rhos.yaml
+   abstract: true
    vars:
      standalone_ceph: true
      featureset_override:
        standalone_environment_files:
          - 'environments/low-memory-usage.yaml'
          - 'ci/environments/scenario001-standalone.yaml'
        tempest_services:
          - aodh
        tempest_run_concurrency: 1
        tempest_tempest_conf_overrides:
          'telemetry.alarm_granularity': '60'
        tempest_test_whitelist:
          - 'tempest.api.identity.v3'
          - 'tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern'
          - 'tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern'
          - 'telemetry_tempest_plugin.scenario.test_telemetry_integration.TestTelemetryIntegration'
  • Create a rhos17 based scenario001 job for intergration pipeline using scenario001 generic job
- job:
    name: periodic-tripleo-ci-rhel-8-scenario001-standalone-rhos-17
    parent: periodic-tripleo-ci-rhel-8-scenario001-standalone
    vars: *vars_rhos17

Changes in component-jobs.yaml

  • Change parent of periodic-tripleo-ci-rhel-8-scenario001-standalone-component job
    • Change parent from periodic-tripleo-ci-rhel-8-scenario001-standalone-rhos-17 to periodic-tripleo-ci-rhel-8-scenario001-standalone(Generic)
# scenario001-standalone-component
- job:
    name: periodic-tripleo-ci-rhel-8-scenario001-standalone-component
    parent: periodic-tripleo-ci-rhel-8-scenario001-standalone ------> (changed from periodic-tripleo-ci-rhel-8-scenario001-standalone-rhos-17)
    abstract: true
    vars: *component_vars
  • Then create version specific component jobs by passing version specific vars.
- job:
    name: periodic-tripleo-ci-rhel-8-scenario001-standalone-component-rhos-17
    parent: periodic-tripleo-ci-rhel-8-scenario001-standalone-component
    abstract: true
    vars: *component_vars_rhos17
  • Then create particular component specific job.
- job:
    name: periodic-tripleo-ci-rhel-8-scenario001-standalone-cloudops-rhos-17
    parent: periodic-tripleo-ci-rhel-8-scenario001-standalone-component-rhos-17
    vars:
      component: cloudops

Alternative Possible Solution

  • Add an anchor on every sceanrio vars on rhos17
  • Reuse them for rhos18/19

https://code.engineering.redhat.com/gerrit/#/c/200373/14/zuul.d/standalone-jobs.yaml

 job:
    name: periodic-tripleo-ci-rhel-8-scenario001-standalone-rhos-17
    parent: periodic-tripleo-ci-rhel-8-standalone-rhos-17
    vars: &scen1_component_vars  ----> Add an anchor here
      standalone_ceph: true
      featureset_override:
        standalone_environment_files:
          - 'environments/low-memory-usage.yaml'
          - 'ci/environments/scenario001-standalone.yaml'
        tempest_services:
          - aodh
        tempest_run_concurrency: 1
        tempest_tempest_conf_overrides:
          'telemetry.alarm_granularity': '60'
        tempest_test_whitelist:
          - 'tempest.api.identity.v3'
          - 'tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern'
          - 'tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern'
          - 'telemetry_tempest_plugin.scenario.test_telemetry_integration.TestTelemetryIntegration'
  • Reuse anchors on new versions says rhos-18(when it comes)
 job:
    name: periodic-tripleo-ci-rhel-8-scenario001-standalone-rhos-18
    parent: periodic-tripleo-ci-rhel-8-standalone-rhos-18
    vars: *scen1_component_vars 

This can create a challenge if in future we decided to drop rhos17 jobs.

What can be done if above situation comes in future?

  • If in future we decide to drop rhos17 jobs then we have to move the detailed vars along with anchor(&scen1_component_vars ) from rhos17 to the next supported release(say rhos18)
Select a repo