Tripleo Tempest Meeting

tags: `Meeting`

Ping list

arxcruz, akhat, chandankumar, mkopec, soniya29, rlandy, ysandeep, kopecmartin,

Liaison: svyas

Agenda

2022-06-29

afuscoar:
- I won't be available for the meeting. I have ansible course at the same time than meeting (June 27, 2022, 3:00pm – July 1, 2022, 11:00pm
- I am on PTO from 2022-07-04 to 2022-07-15.

2022-06-7

What's going on with tempest team?
- use of os_tempest to run tempest in downstream(if it's not using os_tempest)
- need to increase visibility(downstream CI call)
- one week - review meeting
- one week - general meeting
- packing/backporting in tempest..documentation may be a good idea for packing and backporting
- allow list can help cybil to collect tempest data

2022-06-1

Allow-list progress

2022-05-18

Tempest allowed list - progress
- https://review.opendev.org/q/topic:tempest_allow_list
skiplist in imported releases (Ronelle)
- we shouldnt add the stuffs to skiplist if it is concerned with imported releases.
- need to talk this to cix for a new test which is not stable
- imported releases card should stay on blocked whether are not on skiplist.
Downstream bugs - need a check whether the things are still happening
- https://bugzilla.redhat.com/buglist.cgi?bug_status=open&content=openstack-tempest&list_id=12615632&order=Importance&product=Red%20Hat%20OpenStack&query_format=specific

2022-05-04

Tempest allowed list
- https://review.opendev.org/q/topic:tempest_allow_list
- remove tempest_check_whitelist from fs02
- use ALL with releases if we are specifying all the releases
Failing OVB jobs
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/840444
- https://review.rdoproject.org/r/c/testproject/+/42554

2022-04-13

Tempest Scenario manager cleanup
- https://review.opendev.org/q/topic:tempest-scenario-manager-cleanup+owner:+svyas
- git checkout master
- git review -x <id>
Future of Centos Stream support
- https://etherpad.opendev.org/p/qa-zed-ptg
- need volunteers
Tempest allowed list discussion - arxcruz
- postponed to next meeting
CIX
- nothing much it seems

2022-03-16

2022-03-09

Tempest related queries from Pranali Deore, Where to enable srbac related tempest test in tripleo-ci jobs.

2022-03-02

Scenario Manager cleanup
- https://review.opendev.org/c/openstack/barbican-tempest-plugin/+/831364
- https://review.opendev.org/c/openstack/barbican-tempest-plugin/+/829207
Secure-rbac - pdeore
CIX overview
- https://trello.com/c/q8UCTy05/2372-cixlp1962298tripleociproa-tempestscenariotestnetworkbasicopstestnetworkbasicops-tests-are-failing-on-cs-9-jobs-due-to-sshtimeout
  - https://review.opendev.org/c/openstack/tripleo-quickstart/+/830996
- https://trello.com/c/EsgceuAN/2342-cixlp1960310tripleociproa-compute-tests-eg-testrebuildserverwithvolumeattached-failing-with-volume-id-failed-to-reach-available
  - https://review.opendev.org/c/openstack/tempest/+/817635 - waiting for review (the whole queue)

2022-02-09

Tempest updates
- pythontempestconf - https://review.opendev.org/c/openinfra/python-tempestconf/+/820169
- Netstat logging - merged
- consumption of scenario manager - https://review.opendev.org/c/openstack/barbican-tempest-plugin/+/829207
Keystone error
CIX overview
- https://bugs.launchpad.net/tripleo/+bug/1960661

2022-02-09

Tempest updates
- Pythontempest-conf updates
  - https://review.opendev.org/c/openinfra/python-tempestconf/+/820169
- Netstat logging
  - https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/815654
Keystone error
- https://logserver.rdoproject.org/56/36256/47/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/2a1da39/logs/undercloud/var/log/tempest/stestr_results.html.gz
Review list for tripleo-tempest:
- https://review.opendev.org/c/openstack/tempest/+/822318
- https://review.opendev.org/c/openinfra/python-tempestconf/+/820169
- https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/815654
- https://review.opendev.org/c/openstack/tripleo-quickstart/+/769554
  (soniya29): arxcruz, it seems we have merge conflict here and the patch still needs more updation, do you need any help over this like someone to work alongside with you for this very patch?
- https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/720705
  (soniya29): akahat, it seems this patch is pending for too long and there's no update from oct 26, are you working over this or do we require this anymore? Please let us know, if you need any help
CIX overview:

https://trello.com/c/bXBh0xTh/2340-cixlp1959930tripleociproa-tempest-tests-are-all-failing-on-all-ovb-jobs-in-c8-c9-master
- c8 fs01 has passed, let's wait for one more run
- https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master
https://trello.com/c/SfFo0URG/2290-cixlp1957083tripleociproa-multinode-ipa-job-is-not-resolving-the-ntp-hosts-on-some-providers
- https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa
- multinoe-ipa is passing now

2022-01-19

Tempest updates
- Pythontempest-conf updates
  - https://review.opendev.org/c/openinfra/python-tempestconf/+/820169
- Netstat logging
  - https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/815654
- secure rbac fs20/fs01 master/wallaby jobs
  - fs01 master
    - https://bugs.launchpad.net/tripleo/+bug/1956959
    - https://review.opendev.org/c/openstack/tripleo-heat-templates/+/818634 - merged
  - fs01/fs20 wallaby
    -https://bugs.launchpad.net/tripleo/+bug/1956441
Review list for tripleo-tempest:
- https://review.opendev.org/c/openstack/tempest/+/822318
- https://review.opendev.org/c/openinfra/python-tempestconf/+/820169
- https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/815654
- https://review.opendev.org/c/openstack/tripleo-quickstart/+/769554
  (soniya29): arxcruz, it seems we have merge conflict here and the patch still needs more updation, do you need any help over this like someone to work alongside with you for this very patch?
- https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/720705
  (soniya29): akahat, it seems this patch is pending for too long and there's no update from oct 26, are you working over this or do we require this anymore? Please let us know, if you need any help
CIX overview:

https://trello.com/c/TB75nzLv/2288-cixlp1957031tripleociproa-attach-volume-tempest-tests-are-failing-with-failed-to-detach-from-server-with-in-required-time
- (soniya29) have asked whoami-rajat to look over this

Tempest updates
- Pythontempest-conf updates
  - https://review.opendev.org/c/openinfra/python-tempestconf/+/820169
- Netstat logging
  - https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/815654
- secure rbac fs20/fs01 master/wallaby jobs
  - fs01 master
    - https://bugs.launchpad.net/tripleo/+bug/1956959
    - https://review.opendev.org/c/openstack/tripleo-heat-templates/+/818634 - merged
  - fs01/fs20 wallaby
    -https://bugs.launchpad.net/tripleo/+bug/1956441
CIX overview:

https://trello.com/c/TB75nzLv/2288-cixlp1957031tripleociproa-attach-volume-tempest-tests-are-failing-with-failed-to-detach-from-server-with-in-required-time
- (soniya29) have asked whoami-rajat to look over this

2022-01-12

Tempest updates
- Pythontempest-conf updates
  - https://review.opendev.org/c/openinfra/python-tempestconf/+/820169
- Netstat logging
  - https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/815654
- secure rbac fs20/fs01 master/wallaby jobs
  - fs01 master
    - https://bugs.launchpad.net/tripleo/+bug/1956959
      - We are observing this issue because the tests failing here needs to fix the test behaviour for certain scenarios for whom we have updated policy changes in tripleo-heat-templates.
  - fs01/fs20 wallaby
    -https://bugs.launchpad.net/tripleo/+bug/1956441
CIX overview:

https://trello.com/c/TB75nzLv/2288-cixlp1957031tripleociproa-attach-volume-tempest-tests-are-failing-with-failed-to-detach-from-server-with-in-required-time
2. a duplicate, see the comment within the card
https://trello.com/c/3fTSmfNp/2275-cixlp1956233tripleociproa-master-neutrontempestpluginscenariotesttrunktrunktesttesttrunksubportlifecycle-is-failing-runtimeerror
4. this is a neutron's bug and it has a fix already, how is this related to tempest?

2022-01-05

Openstack-health gets a new volunteer - arxcruz
Tempest updates
- [WIP] Python-tempestconf progress
  - https://review.opendev.org/c/openinfra/python-tempestconf/+/820169
- [WIP] Netstat logging
  - https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/815654
- secure-rbac wallaby jobs
  - https://review.rdoproject.org/r/c/rdo-jobs/+/36875 - merged
- updates ipv6 network within tempest - ysandeep
CIX overview
- https://trello.com/b/j4IcIomh/production-chain-escalation
2022 plans

2021-12-15

Tempest updates
- Python-tempestconf
  - https://review.opendev.org/c/openinfra/python-tempestconf/+/820169
- secure-rbac wallaby jobs
  - https://review.rdoproject.org/r/c/rdo-jobs/+/36875
  - we have got zuul success, do we need anything else here to get this merged.
  - pass links to martin of this stuffs
- (Query: sandeep)Is it possible that tempest create ipv6 network instead of ipv4 network?
  - https://bugs.launchpad.net/tempest/+bug/1401726
  - we can create this in tempest
CIX overview
- https://trello.com/b/j4IcIomh/production-chain-escalation

2021-12-01

No agenda for today's call - skipping today's call

<ysandeep|ruck> kopecmartin, arxcruz|rover: hey o/ soniya is on leave today, If you have any agenda for today's tempest meeting please add on: https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ , Depending on whether we have anything on agenda - we will decide whether to meet/skip.
<arxcruz|rover> ysandeep|ruck: i don't have anything 
<ysandeep|ruck> arxcruz|rover, thanks! lets wait for kopecmartin 
<kopecmartin> i don't have anything 
<ysandeep|ruck> kopecmartin, arxcruz|rover ack, lets skip today's call then.
<kopecmartin> ack

Agenda

2021-11-24

(Sandeep)
- Have you seen this email about Zuul Config Errors from openinfra/python-tempestconf.
  - http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025797.html
- When using custom overcloud name - overcloud1 instead of overcloud - tempest related tests are failing - we don't have any public network configured/discovered by python-tempestconf
  - https://logserver.rdoproject.org/54/31954/13/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset064-master/708e5b3/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz
  - Diff with a non custom overcloud name: http://pastebin.test.redhat.com/1010571
Verify the project personas for OSP 17 (patrole or selective tempest tests from upstream to verify read-only functionality)
RBAC failures - need to start creating bugs for (master, wallaby, rhos-17)
Cleanup skiplist
- Tests have been added long back and they are not cross-verified if they are required in skiplist.
- please review the following patches
  - https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/818916

CIX overview

https://trello.com/b/j4IcIomh/production-chain-escalation

Agenda

2021-11-10

Netstat logging updates
Updating python-tempest conf
- gates are failing
  - https://review.opendev.org/c/openinfra/python-tempestconf/+/814373
tempest scenario tests failing on rhos-17 line
Tempest allowed tests
- https://review.opendev.org/q/topic:"tempest-allow-minimal"+(status:open OR status:merged)
CIX time
- https://trello.com/b/j4IcIomh/production-chain-escalation
what is the status of TripleO Health - is it full replacement of OpenStack Health?
irc_nick name:- frenzy_friday

2021-10-27

Netstat logging
- no updates
Updating python-tempest conf
- needs a discussion with glance team regarding which parameters to be populated with python tempestconf
Retiring of Patrole(still on hold)
- There has been a lot of failured in the patrole gates in past few cycles
- Not much organizations have shown requirement with it
- agred with gmann to wait with this during this cycle - we'll revisit this during the next PTG
Tempest allowed tests
- https://review.opendev.org/q/topic:"tempest-allow-minimal"+(status:open OR status:merged)
whether to use os-tempest in downstream or not starting from OSP-17?
- https://docs.google.com/document/d/1m8EBdNjOY8FvgCZkjiim5F08RTfH9eaqDGuMlukI2MA/edit?usp=sharing
- (martin) no updates, the group is busy with redefining the jobs currently
Need reviews on Improve cleanup after Swift testing
- https://review.opendev.org/c/openstack/tempest/+/812455

2021-10-12

patrole updates
- tests passes with patrole reader job but fails with patrole-admin and patrole-member
- https://review.opendev.org/c/openstack/patrole/+/795548
- https://review.opendev.org/c/openstack/patrole/+/807131
Tempest client required for patrole
- https://review.opendev.org/c/openstack/tempest/+/810015
Netstat logging
- https://review.opendev.org/c/openstack/tempest/+/813590
Tempest allowed tests
- Minimal featureset is on the way
  - https://review.opendev.org/q/topic:"tempest-allow-minimal"+(status:open OR status:merged)
- Integration of allowed list
whether to use os-tempest in downstream or not starting from OSP-17?
- https://docs.google.com/document/d/1m8EBdNjOY8FvgCZkjiim5F08RTfH9eaqDGuMlukI2MA/edit?usp=sharing
Need reviews on Improve cleanup after Swift testing
- https://review.opendev.org/c/openstack/tempest/+/812455

2021-10-06

Tempest
- reviews required - https://review.opendev.org/c/openstack/tempest/+/810015
patrole updates
- needs discussion for the confusions with following things:
  - Why is apis defined here - https://docs.openstack.org/api-ref/network/v2/index.html?expanded=delete-a-floating-ip-port-forwarding-detail,list-loggable-resources-detail are different from their respective neutron extensions.
  - for e.g - we have api named loggable resource as above and name of its respective neutron extension is logging resource
  - Same thing with log resource
  - Do we need to correct this in the api-ref documentation or neutron side?
Tempest allowed tests
- Minimal featureset is on the way
  - https://review.opendev.org/q/topic:"tempest-allow-minimal"+(status:open OR status:merged)
- Integration of allowed list
whether to use os-tempest in downstream or not starting from OSP-17?
- https://docs.google.com/document/d/1m8EBdNjOY8FvgCZkjiim5F08RTfH9eaqDGuMlukI2MA/edit?usp=sharing

2021-09-29

Tempest
- Addition of floating_ips-port-forwarding client
- https://storyboard.openstack.org/#!/story/2002641
patrole gate updates
- gates broken due to frequent failure of test_force_detach_volume_from_instance - done!
- https://review.opendev.org/c/openstack/patrole/+/800594
Tempest allowed tests
- Minimal featureset is on the way
  - https://review.opendev.org/q/topic:"tempest-allow-minimal"+(status:open OR status:merged)
- Integration of allowed list
whether to use os-tempest in downstream or not starting from OSP-17?
- https://docs.google.com/document/d/1m8EBdNjOY8FvgCZkjiim5F08RTfH9eaqDGuMlukI2MA/edit?usp=sharing

2021-09-22

Tempest
- [WIP] Addition of floating_ips-port-forwarding client
- https://storyboard.openstack.org/#!/story/2002641
patrole gate updates
- gates broken due to frequent failure of test_force_detach_volume_from_instance
- https://review.opendev.org/c/openstack/patrole/+/800594
Tempest allowed tests
- https://review.opendev.org/q/topic:"tempest-allow"+(status:open OR status:merged)
- Minimal featureset is on the way
  - https://review.opendev.org/q/topic:"tempest-allow-minimal"+(status:open OR status:merged)
whether to use os-tempest in downstream or not starting from OSP-17?
- https://docs.google.com/document/d/1m8EBdNjOY8FvgCZkjiim5F08RTfH9eaqDGuMlukI2MA/edit?usp=sharing

2021-09-15

Tempest
- [WIP] Addition of floating_ips-port-forwarding client
- https://storyboard.openstack.org/#!/story/2002641
patrole gate updates
- gates broken due to frequent failure of test_force_detach_volume_from_instance
- https://review.opendev.org/c/openstack/patrole/+/800594
Tempest allowed tests
- https://review.opendev.org/q/topic:"tempest-allow"+(status:open OR status:merged)
- Minimal featureset is on the way
  - https://review.opendev.org/q/topic:"tempest-allow-minimal"+(status:open OR status:merged)
secure-RBAC
- Lance have produced custom policy file to enable project personas
- Each DFG to review their portion of the proposed custom policy overrides - https://docs.google.com/document/d/1UXkmwsftO5fI50u4Kp4MEpFS3ngNwpS5nVeXL17U3qQ/edit#heading=h.ybpnpzl9o2tc
- Each DFG to consider if there is an existing TripleO CI scenario they can use to test their custom policy changes
- Need volunteer to work with Lance for integrating the above template into common scenario to run with tempest
- devstack job: https://review.opendev.org/c/openstack/devstack/+/804263
- editing tempest jobs to use rbac: https://review.opendev.org/c/openstack/tempest/+/614484
whether to use os-tempest in downstream or not starting from OSP-17?

2021-09-08

cs9 hashtag for changes related to CenOS Stream 9
- https://review.opendev.org/q/hashtag:"cs9"+(status:open OR status:merged)
Html file not rendered
- https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-scenario010-standalone-network-rhos-17/771a3d9/logs/undercloud/var/log/tempest/
- Good
  https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-scenario010-standalone-network-rhos-17/05864c0/logs/undercloud/var/log/tempest/stestr_results.html
Patrole updates
- Regarding patrole cores
  - http://lists.openstack.org/pipermail/openstack-discuss/2021-September/024728.html
- gate issues still going on
  - https://review.opendev.org/c/openstack/patrole/+/660120
  - https://review.opendev.org/c/openstack/patrole/+/802549
Tempest allowed tests
- https://review.opendev.org/q/topic:"tempest-allow"+(status:open OR status:merged)
- Minimal featureset is on the way
  - https://review.opendev.org/q/topic:"tempest-allow-minimal"+(status:open OR status:merged)

2021-09-01

patrole gates broken
- gmann has proposed the patch for the same - https://review.opendev.org/c/openstack/patrole/+/802549
patrole updates:
Tempest client updates:
- https://review.opendev.org/c/openstack/tempest/+/799815
Tempest allowed tests
- https://review.opendev.org/q/topic:"tempest-allow"+(status:open OR status:merged)
periodic featureset 35 wallaby times out running tempest (2 hours)[promotion blocker]
- https://bugs.launchpad.net/tripleo/+bug/1939023/
  - https://review.opendev.org/q/I9d0ddcc88098a5b891829192f1ce656842d0aa15
bump tempest to version 28.0.0 in
- wallaby: https://review.rdoproject.org/r/c/rdoinfo/+/34917
- victoria: https://review.rdoproject.org/r/c/rdoinfo/+/33101
- ussuri: https://review.rdoproject.org/r/c/rdoinfo/+/33102
- train: https://review.rdoproject.org/r/c/rdoinfo/+/35113
Scenario manager effort
- probably this stuffs needs to be started in next cycle.
- https://etherpad.opendev.org/p/qa-yoga-ptg
  - I created a topic in QA's PTG agenda

2021-08-18

patrole gates broken
- gmann has proposed the patch for the same - https://review.opendev.org/c/openstack/patrole/+/802549
patrole updates:
Tempest client updates:
- https://review.opendev.org/c/openstack/tempest/+/799815
Tempest allowed tests
- https://review.opendev.org/q/topic:"tempest-allow"+(status:open OR status:merged)
periodic featureset 35 wallaby times out running tempest (2 hours)[promotion blocker]
- https://bugs.launchpad.net/tripleo/+bug/1939023/
  - https://review.opendev.org/q/I9d0ddcc88098a5b891829192f1ce656842d0aa15
bump tempest to version 28.0.0 in
- wallaby: https://review.rdoproject.org/r/c/rdoinfo/+/34917
- victoria: https://review.rdoproject.org/r/c/rdoinfo/+/33101
- ussuri: https://review.rdoproject.org/r/c/rdoinfo/+/33102
- train: https://review.rdoproject.org/r/c/rdoinfo/+/35113
Scenario manager effort

2021-08-11

patrole gates broken
- patches suffer due to the gate issues
- discussed in last qa meeting
- needs to be fixed - since it is blocking the patches over patrole
Tempest allowed tests
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/792575
periodic featureset 35 wallaby times out running tempest (2 hours)[promotion blocker]
- https://bugs.launchpad.net/tripleo/+bug/1939023/
bump tempest to version 28.0.0 in wallaby
- https://review.rdoproject.org/r/c/rdoinfo/+/34917

2021-08-04

New bug in Queens: tempest run: error: unrecognized arguments: –exclude-list /home/zuul/tempest/etc/tempest_excludelist.txt –include-list /home/zuul/tempest/etc/tempest_includelist.txt
- https://bugs.launchpad.net/tripleo/+bug/1933922
  - Fix: https://review.opendev.org/c/openstack/tripleo-ci/+/798827: Abandoned
  - backports are merged
RBAC tests progress
- no updates for now
Tempest skiplist updates
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/789625
designate_tempest_plugin
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798390
Tempest allowed tests
- https://review.opendev.org/c/openstack/tripleo-ci/+/792576

2021-07-07

New bug in Queens: tempest run: error: unrecognized arguments: –exclude-list /home/zuul/tempest/etc/tempest_excludelist.txt –include-list /home/zuul/tempest/etc/tempest_includelist.txt
- https://bugs.launchpad.net/tripleo/+bug/1933922
  - Fix: https://review.opendev.org/c/openstack/tripleo-ci/+/798827
- backports (merged already):
RBAC tests progress
- Log resource client
- RBAC tests for the log reosurce
  - https://review.opendev.org/c/openstack/patrole/+/795548
Tempest skiplist updates
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/789625
designate_tempest_plugin
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798390
- Abandoned by ronelle and we are moving we the wes's patch
Tempest allowed tests
- https://review.opendev.org/c/openstack/tripleo-ci/+/792576

2021-06-30

New bug in Queens: tempest run: error: unrecognized arguments: –exclude-list /home/zuul/tempest/etc/tempest_excludelist.txt –include-list /home/zuul/tempest/etc/tempest_includelist.txt
- https://bugs.launchpad.net/tripleo/+bug/1933922
  - Fix: https://review.opendev.org/c/openstack/tripleo-ci/+/798827
- backports:
- mkopec to check if downstream will pull those changes ^^
  - if not, we can merge them
RBAC tests progress
- Log resource client
  - tempest-multinode-full-py3 job is failing because of test_live_migration_with_trunk test
  - nova has reported an issue about it
  - https://238189cb2a75413f1bcf-1d67ec5eb204f28641a7073258861323.ssl.cf2.rackcdn.com/796842/4/check/tempest-multinode-full-py3/4e9cf52/testr_results.html
- RBAC tests for the log reosurce
  - https://review.opendev.org/c/openstack/patrole/+/795548
Tempest skiplist updates
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/789625
designate_tempest_plugin
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798390

2021-06-23

RBAC test discussion - svyas
- Tempest client is ready for log-resource
  - https://review.opendev.org/c/openstack/tempest/+/796842
  - Patch needs testcases and other modifications..its in progress
- https://hackmd.io/rBqcts18Rc-qCOqe1DM1gg
Tempest skiplist factoring
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/789625
Tempest keystone tests done!

2021-06-16

Retiring x/mogan, x/valet, x/kingbird
- https://opendev.org/openstack/tempest/src/commit/663787ee794df54e7ded41e5f3e8ae246e9b4288/tools/generate-tempest-plugins-list.py#L59
- https://review.opendev.org/c/x/mogan/+/767718
- https://opendev.org/x/mogan/commits/branch/master
- they will skipped since unmaintained
RBAC test discussion
- needs certain discussion, let's follow up on openstack-discuss ML
- https://hackmd.io/rBqcts18Rc-qCOqe1DM1gg
Tempest skiplist factoring
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/789625

2021-06-09

Tempest skiplist factoring
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/789625
Adding RBAC tests for the policy actions
- https://storyboard.openstack.org/#!/story/2002641
- https://docs.openstack.org/api-ref/network/v2/#log-resource

2021-05-19

stestr results formatting fixed
Tempest skiplist factoring
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/789625
Fix train c7 job on os_tempest
- https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/789830
connectivity issue timeout
Adding tox py39 jobs to tempest -WIP
- (kopecmartin) what exactly is WIP here? we've already added a new zuul template which runs unit tests with py39 .. or is there something else I am not aware of?
- no its done!

2021-05-12

who is going to fix tempest results formatting?:
- https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-tripleo-wallaby/b63c8b2/logs/undercloud/var/log/tempest/stestr_results.html.gz
- may be file not rendered..
- when saved to a file and opened locally it's rendered correctly
- needs to discuss with #rhos-ops and #rdo
Tempest skiplist refactoring
- Differences between releases upstream and downstream
Tempestconf updates on other releases
- needs discussion with martin
- 3.2.1 tag has been released, only rdoinfo update is needed in order to get that tag to train
- train is updated (not to the new tag, but to a commit - the same result) - https://review.rdoproject.org/r/c/rdoinfo/+/33572
Fix train c7 job on os_tempest
- its failing
- update tempest code on train release [update tempest rpm for train]
  - tempest has been bumped to 26.1.0 in train https://review.rdoproject.org/r/c/rdoinfo/+/33420
- https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/789830 - NV job
openstack/tempest listed twice in RDO zuul
https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/rdo.yaml#L696
https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/rdo.yaml#L1114
which is correct?
https://review.rdoproject.org/r/c/config/+/33588

2021-05-05

Tempest skiplist refactoring

Differences between releases upstream and downstream

Points to ponder

  - should there be global kind of jobs only or jobs per release?
  - empty array is dangerous since it skips everything
  - can we define combination upstream release and internal release

Tempestconf updates on other releases
- needs disussion with martin
- 3.2.1 tag has been released, only rdoinfo update is needed in order to get that tag to train
Fix train c7 job on os_tempest
- its failing
- update tempest code on train release[ update tempest rpm for train]
- https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/789830 - NV job
openstack/tempest listed twice in RDO zuul
https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/rdo.yaml#L696
https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/rdo.yaml#L1114
which is correct?
https://review.rdoproject.org/r/c/config/+/33588

2021-04-29

Updates from PTG
Cirros download from other sources
- updates - done!
- add cirros 5.2 image..merged ad working
Tempest tests consolidation in tempest-skiplist project
- gather all the tests..
- requires some more work
Upstream updates
- Patrole stuffs
- consumption of scenario manager within plugins
  - need to wait until train is in EM
  - low priority for now..we can rethink over it
- updation of tempest tox jobs to py39…needs discussion
  - needs to wait until centos stream is in Y cycle
glance-tempest-plugin
- https://review.rdoproject.org/r/q/topic:"add-glance-tempest-plugin"+(status:open OR status:merged)
- https://review.opendev.org/q/project:+openstack/glance-tempest-plugin
Update discrete jobs with latest tempest configuration(soniya29)
- https://review.rdoproject.org/r/c/rdo-jobs/+/33512
tempest 26.1.0 for Train - https://review.rdoproject.org/r/c/rdoinfo/+/33420
- do we still support centos-7 jobs as they're failing for not related reasons in ^^, looks like they are not maintained (they aren't executed regularly)
please review collect-logs patch: https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/783843

2021-04-07

Cirros download from other sources
Tempest allowed list
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/782253
Tempest tests consolidation in tempest-skiplist project
Upstream updates
- no updates
glance-tempest-plugin
- https://review.rdoproject.org/r/q/topic:"add-glance-tempest-plugin"+(status:open OR status:merged)
Update discrete jobs with latest tempest configuration(soniya29)
- https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/vars/redhat-8.yml#L36
- https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/distgit-jobs.yaml

2021-03-31

Cirros download from other source
- updates?
Tempest allowed list
- - https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/782253
Tempest tests consolidation in tempest-skiplist project
- Can we put in the PTG topics?
Upstream updates
- Scenario-Manager
  - Tempest stable version 27.0.0 is released and we have stable interface
- Patrole
Bump tempest version 27.0.0. in rdo
- Not needed for Wallaby as tempest is not pinned yet https://github.com/redhat-openstack/rdoinfo/blob/master/tags/wallaby.yml
- Victoria and Ussuri is TBD
- Train - kopecmartin hasn't had time to bump to 26.1.0 yet
glance-tempest-plugin
- chandan will ping pranali
Update discrete jobs with latest tempest configuration(soniya29)
- https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/vars/redhat-8.yml#L36
- https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/distgit-jobs.yaml

2021-03-24

Cirros download from other source
- https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/778157
- https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/782321
- Cirros from cached image
Tempest allowed list
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/782253
Tempest tests conslidation in tempest-skiplist project
Upstream updates
- Scenario Manager updates:
  - https://review.opendev.org/c/openstack/tempest/+/781616
- Patrole:
  - started locally.
glance-tempest-plugin
- https://review.opendev.org/q/project:+openstack/glance-tempest-plugin
Update discrete jobs with latest tempest configuration
- e.g tempest plugin to tempest services
- run more jobs against tempest spec file changes
- https://review.rdoproject.org/r/c/rdoinfo/+/32388
- https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/distgit-jobs.yaml

2021-03-10

Cirros download from other source
- https://review.opendev.org/c/osf/python-tempestconf/+/775195
- its on going- arx cruz will be working on it
Cirros from cached image
- https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/778157
Tempest tests conslidation in tempest-skiplist project
- Did not started yet
tempest container
- https://review.opendev.org/q/topic:"remove-tempest-container"+(status:open OR status:merged)
Upstream Updates
- tempest 26.1.0 has been released
- I need to propose patches to RDO to bump the version there (mkopec)
glance-tempest-plugin
- Integrating with tripleo-ci
  - packaging in rdo
  - integrating with os_tempest
  - enabling it in specific jobs
Paused Items
- stestr
- tempest cleanup
- ironic os_tempest
- infrared integration with skiplist tooling
- patrole

2021-03-03

Redesign of tempest ironic resources setup - akahat
- https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/720705/
- akhat wishes to work..needs discussion
- redesign needed
- promotor work has priority . so akahat will work on spare time
Cirros download from other source
- https://review.opendev.org/c/osf/python-tempestconf/+/775195
- its on going- arx cruz will be working on it
Cirros from cached image
- https://review.opendev.org/c/openstack/tripleo-ci/+/778155
- https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/778157
Tempest tests conslidation in tempest-skiplist project
- Did not started yet
tempest container
- https://review.opendev.org/q/topic:"remove-tempest-container"+(status:open OR status:merged)
Upstream Updates
- New tempest tag
  - cinder tempest plugin and tempest tag update for rhos-16
- Scenario Manager Updates
  - https://review.opendev.org/q/topic:+bp/tempest-scenario-manager
Paused Items
- stestr
- glance tempest plugin
- tempest cleanup
- ironic os_tempest
- infrared integration with skiplist tooling
- patrole

2021-02-24

Moving all the tempest tests into skip list
- https://hackmd.io/Q6n4Dv7FToq4EZGmqVeYBQ
  - no updates for now
- Remove tests from commandline: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/772319
New Upstream tag update
- Probably new week
  - Need to update till train for c8?
    - Test params changes to make it more inclusive ?
    - didnt break anything
    - not sure possible now
Cirros download from other source
- https://review.opendev.org/c/osf/python-tempestconf/+/775195
- Alternative path for cirros download
- arxcruz- new option :- retry [is implemented]
- arxcruz: will try to remove all tempest variables. no hardcoded on tempest playbook
- check for the path then go for the source 1, source 2, etc
Scenario Manager:
- Patches lined up for reviews:-
patrol
- https://storyboard.openstack.org/#!/story/2005489
RBAC updates
Paused Items
- stestr
- glance tempest plugin
- tempest cleanup
- infrared integration with skiplist tooling

2021-02-03

Tempest tests optimization ( targeting tests to services)
- Feeback received from community call
  - It requires revisiting
  - Two things to ponder for optimisation:-
    - the tests management
    - specific tests to be run in the job for optimisation
  - each scenario contains certain components so let's ask DFGs for their inputs
skiplist update
- https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/772319
Updates from upstream
- Scenario Manager
  - https://review.opendev.org/c/openstack/tempest/+/773783
  - https://review.opendev.org/c/openstack/tempest/+/773019
- Patrole stable maintainance
Patches needing attention
Paused Items
- stestr
- glance tempest plugin
- tempest cleanup
- infrared integration with skiplist tooling
  - arxcruz/mkopec will start working on this action item
  - its a matter of convenienance of people for skiplist tool
tempest container deprecration / removal status?

chandankumar will be taking care of this

Other updates:-
- glance-tempest-plugin

2021-01-27

Upstream initiatives
- Secure RBAC
  - Meeting updates
- Scenario Manager
Patches needing attention
- https://review.rdoproject.org/r/#/c/31502/ - python-tempestconf's bump for train
- https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/771416 - Neutron / rc file
  - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_821/770767/5/check/tripleo-ci-centos-7-undercloud-oooq/821acb9/logs/undercloud/var/log/tempest/stestr_results.html
- os_tempest undercloud
Tempest container
- http://lists.openstack.org/pipermail/openstack-discuss/2021-January/020059.html
- Revisit new week
os_tempest infrared integration
- integrate skiplist with infrared first
skiplist tooling
- Add test merged
- Remove test in progress
  - https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/772319
glance-tempest-plugin
Paused Items
- stestr
- tempest cleanup

2021-01-20

Upstream initiatives
- Secure RBAC
  - meeting with Hary, Lance for the further discussion
- Patrole
  - no updates
- Scenario Manager
  - https://review.opendev.org/c/openstack/tempest/+/766472
  - https://review.opendev.org/c/openstack/tempest/+/766015
    - (mkopec) I left a few notes under "Investigating if the following methods can loose '_' (private) prefix" in https://etherpad.opendev.org/p/tempest-scenario-manager
python-tempestconf bump to 3.1.0 in RDO - https://review.rdoproject.org/r/#/c/31502/
- (mkopec) the bump is ok in my opinion, I just can't verify it because tripleo-ci jobs are failing for not bump related reasons, need advice on the jobs from someone from tripleo-ci
Skiplist tooling updates - arxcruz
- updates?
- ready for review. it will work with tempest plugin if skiplist is installed
glance tempest plugin
- https://review.opendev.org/plugins/gitiles/openstack/glance-tempest-plugin/
- https://opendev.org/openstack/glance-tempest-plugin
- the project doesn't seem active much:
  - https://review.opendev.org/q/project:openstack/glance-tempest-plugin+(status:open+OR+status:merged)
  - (soniya) I will contact pranali and abhishekh kekane(glance PTL) regarding this
Tempest clean up implementation- needs discussion with chandan
- mkopec can help in reviewing patches but proposing patches is not possible.
- Voluteers are welcomed here
Updates on stestr timing data - mkopec
- (mkopec) - I have a plan, however, I don't have time for this atm

2021-01-13

Upstream initiatives
- Secure RBAC
  - http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018800.html
- Stable version of Patrole
- Stable version of scenario Manager
  - cores to audit the etherpad this week probably
- diversity language:
  - https://review.opendev.org/q/topic:"inclusive_jargon"+(status:open OR status:merged)
  - have more diverse language
python-tempestconf bump to 3.1.0 in RDO - https://review.rdoproject.org/r/#/c/31502/
- when trying to test if the bump is safe, the jobs are failing unrelated to the bump
tempest role share w/ infrared group - Kopecmartin
- if the os_tempest role is meant to replace downstream one then it is a tough goal
  - the downstream infrared plugin running tempest contains too much custom code
- Let's start with tempest-skiplist
  - making tempest-skiplist an infrared plugin so that we can try to get it integrated downstream is much more realistic goal so let's start with that and then we'll reevaluate the rest
Skiplist tooling updates - arxcruz
- No news, still handling os_tempest everywhere post merge issues
Other

2020-12-09

glance tempest plugin
- https://github.com/PranaliDeore/glance-tempest-plugin
Tempest updates
- Patrole
- Scenario Manager
Tempest clean up implementation
- mkopec can help in reviewing patches but proposing patches is not possible.
- Voluteers are welcomed here
Questions: Cancelling tempest meeting due to holidays?

2020-11-25

Enabling tempest clean up in TripleO ci job
- https://review.opendev.org/762405 is merged now
  - Updates and upgrade job in periodic
    - tempest needs to be turned back on
  - fs002, fs020 and standalone api & scenario
Updates on stestr timing data
- (mkopec) so far none updates
Testing different image apart from cirros in tripleo ci job
- Amol will take care of that
Any other updates on which the team is working on wallaby goals
in order to merge https://review.opendev.org/c/osf/python-tempestconf/+/762736 we need a new dependency in the spec https://review.rdoproject.org/r/#/c/31202/

2020-11-18

Bunch of patches to review for tempestconf
- https://review.opendev.org/#/q/owner:mkopec%2540redhat.com+status:open+project:osf/python-tempestconf
- Needs discussion - https://review.opendev.org/762707
- tempest cleanup ostempest - https://review.opendev.org/762405
- increase tempest run effectivness by preserving stestr's timing data
  - POC: https://github.com/kopecmartin/ansible-role-job-data
    - requires a small edit in infrared: https://review.gerrithub.io/c/redhat-openstack/infrared/+/505741
    - example of the stestr's timing data: https://github.com/kopecmartin/job-data-repo/tree/master/jobs/rhos-timing-test-16.1-full-slow
    - https://projects.engineering.redhat.com/browse/RHOSINFRA-3703
    - https://github.com/kopecmartin/job-data-repo
  - discussed points:
    - Store the previous job data .stestr specific file in a git repo and periodically update it
    - the stestr's files don't have to be stored after each tempest run
    - we could let ansible-role-collect-logs to log the stestr's files

2020-11-11

Involvement in patrole maintaince for victoria release
- https://bugzilla.redhat.com/show_bug.cgi?id=1533574
- this bug will be closed
Asking for feedback on PTG - send the doc to livnat
svyas and mkopec to collaborate on this
os_tempest patches status
- https://review.opendev.org/#/c/759830/3
[openstack][qa][tempest] tempest cleanup ansible role - http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018634.html
- Can we include it in os_tempest role?
  - mkopec will check th code and inform back on how it can be included
  - if you wanna include it in the gates, just edit gate job playbooks similarly like we did in tempest -> https://review.opendev.org/#/c/698610/19/playbooks/devstack-tempest.yaml
    - basically just call tempest-cleanup role before and after running tempest tests
  - or put one task before 'tempest run' here https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/tasks/tempest_run.yml and one after - the first one will create saved_state.json and the second will clean any leftover resources
    - make it conditional and enable it the gates
    - https://review.opendev.org/762405
      - adds support for tempest cleanup
TODO(Soniya), ping slawq for neutron ci debugging training
octavia tempest plugin - chandankumar
karlos not going to it

2020-11-04

skiplist for undercloud is fixed now
Updates from PTG:
- Tempest: https://etherpad.opendev.org/p/qa-wallaby-ptg
  - patrole stable release require maintainers
    - https://etherpad.opendev.org/p/qa-wallaby-ptg Line: 138
  - scenario manager work
    - https://etherpad.opendev.org/p/tempest-scenario-manager
- OSA : https://etherpad.opendev.org/p/osa-wallaby-ptg Line:131

Drop resource creation tasks out of os_tempest - OSA and TripleO manage resource creation themselves and pass required vars to os_tempest for config generation - go and talk with triplo team and try to come to agreement one more time. Probably we shouldn't push it too much.
https://review.opendev.org/733892
https://review.opendev.org/733894

os_tempest everywhere
- https://review.opendev.org/#/c/759830/3 and https://review.opendev.org/#/c/746509/30
https://bugs.launchpad.net/tripleo/+bug/1900357
ssh timeout issue
*

2020-10-23

os_tempest everywhere
- https://review.opendev.org/#/c/746509/
  - Need fixes on upgrade job side
- os_tempest centos-7 job
  - https://review.opendev.org/#/c/758823/
Tempest SKip list for undercloud
- Enable support to multiple deployments - https://review.opendev.org/#/c/759271/
- Add support for skiping tests based on deployment type - https://review.opendev.org/759211
- Need to update skip list for rocky & queens
- Need to add a retro card for above design miss
- release note page throws 404 -> https://docs.openstack.org/releasenotes/openstack-tempest-skiplist
  - if we want release notes, need to add a zuul job for that
    - i'd say it's good to have this for bigger projects, not sure if skiplist is such
  - if not, let's remove it (tox, Readme …)
  - https://review.opendev.org/#/c/759409/

2020-10-14

News: tempest-25.0.0 is now consumed from train onwards
Work on merging Deprecate validate-tempest patches
Convert tempest.yaml tasks file into playbook and
call via tags using quickstart.sh and wire up with tripleo-ci
- https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/playbooks/tasks/tempest.yml
- Just like validate-tempest
- currently not so critical
Need to assign someone to work on cirros mess
- Work with upstream infra to find out whether cirros image is available
  in upstream nodepool images used in tripleo-job
  - if not, add the images
- fix tempest.yaml playbook to consume image from there
- Remove other reference of cirros image defined at multiple places
- <arxcruz> upstream wont operations inside image
- <wes> two problems: getting cirros image on nodepool, variables configuring across
- complete os_tempest roll out across all branches
- take a look at upgrade jobs
- consolidate variables
- then go for cirros image

2020-10-07

Deprecare validate-tempest
- Use os_tempest everywhere
  - https://review.opendev.org/746509
  - Blockers?
    - Reviews and random failures
Tempest-skiplist - Add test command with lasers
- https://review.opendev.org/#/c/754994/
Os_tempest upgrade bug
- https://bugs.launchpad.net/tripleo/+bug/1896126
  - https://review.opendev.org/#/c/753393/
    - Blocked by https://review.opendev.org/#/c/733408/
Bump tempest-25.0.0
- Blockers? - none, just review of the patches below is needed
- https://review.rdoproject.org/r/#/c/29616/
- https://review.rdoproject.org/r/#/c/29612/
- https://review.rdoproject.org/r/#/c/29611/
Future of tempest container
- https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view#2020-06-19

2020-09-23

validate-tempest removal
- use os_tempest everywhere:
- Os_tempest upgrade bug
  - https://bugs.launchpad.net/tripleo/+bug/1896126
    - https://review.opendev.org/#/c/753393/
- Further action items:
  - ~~Sync tests to openstack-tempest-skiplist for older release(queens, rocky, stein, train)~~
  - cleanup featureset files
  - Deprecate validate-tempest
    - Donot forgot to add relase notes
  - if a neutron parent class fails, then add the feature to rewrite the child tests with same reason in the skip list file
    - To make it easier to add or remove tests based on criteria.
    - Also help in adoption in infrared and jenkins
PTG planning and discussion items
- qa topics for PTG:
  - https://etherpad.opendev.org/p/qa-wallaby-ptg
  - not many topics there, feel free to add anything to discuss there
- Soniya and Martin
  - Soniya will be discussing on scenario manager work
    - https://etherpad.opendev.org/p/tempest-scenario-manager
- Arx to present os-tempest-skiplist during ptg and discuss on consumption and also in openstack-ansible ptg.

2020-09-16

os_tempest for rocky and queens
- https://review.opendev.org/746509
- Testing here: https://review.rdoproject.org/r/#/c/28988/
  - For rocky
    - tempest_private_net_provider_type: "vlan" is not getting setted
  - for queens
    - hitting while creating tempest projects

keystoneauth1.exceptions.connection.SSLError: SSL exception connecting to https://10.0.0.5:13000/v3/auth/tokens: HTTPSConnectionPool(host='10.0.0.5', port=13000): Max retries exceeded with url: /v3/auth/tokens (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)'),))

ironic tempest plugin update
- https://review.opendev.org/#/c/720705/
scenario manager update
Cockpit with skipped tests
- https://review.rdoproject.org/r/#/c/29352/ WIP
ansible-role-collect-logs review:
- https://review.opendev.org/#/c/751661/

2020-08-21

tempestconf-3.0.2 is now updated for train - https://review.rdoproject.org/r/28818
os_tempest everywhere
- https://review.opendev.org/#/c/746509/
Ironic update
- ironic tempests tests are running now against OSA os_ironic
  - https://review.opendev.org/736507
- os_tempest ironic integration - https://review.opendev.org/720705
Scenario Manager update
Move this meeting to wednesday??

2020-08-07

Using os_tempest on queens
- Set proper provider type for queens - https://review.opendev.org/744447
Ironic tempest plugin update
- Added ironic specific variables in the featureset060 file - https://review.opendev.org/731704
- Adding ironic chessis_node_list test to skiplist - https://review.opendev.org/744882
- [Questions] Since Ironic team is not interested in multinode ironic testing, how to test it?
  - Added tempest ironic resources setup - https://review.opendev.org/720705
tempestconf 3.0.1 release for train
- Added workaround for handling mock py2 & py3 issue - https://review.rdoproject.org/r/28823
- Update python-tempestconf to 3.0.2 in Train - https://review.rdoproject.org/r/28818
scenario manager update
loci source code?

2020-07-31

updates on skip list production
- os_tempest for stein and queens - https://review.opendev.org/742624
- Are we still facing issue on ci.centos related to skip list installation?
  - https://review.opendev.org/#/c/744097/ venv patch
    - move it to install_deps and set python2 & 3 in quickstart.sh script -> weshay needs to rethink
- improvements to skip list
  - add a validation if the test is duplicated on the skiplist
    - Since we have one single file, a single test can be skipped.
cirros cleanup (this sprint task)? -> once os_tempest done
scenario manager refactoring updates
- https://review.opendev.org/#/c/724631/
Tempest cli coverage
containers collect logs update
- collection time from 6 to 12 mins
  - from one node, log collection time is 6 min
  - from all nodes - 12 mins
  - and for uploading 5 mins
  - Explore around these areas to improve timing -> make it to 10 -15 mins
Ironic tempest plugin update

2020-07-24

Updates on skip list production
- periodic jobs for skiplist
  - https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/tempest-skiplist-jobs.yaml
    - Needs to be communicated to the team to ignore these jobs
  - Final review for pipeline: https://review.rdoproject.org/r/28650
    - needs reviews
- single playbook for tempest fun: https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/playbooks/tempest.yml
- os_tempest for stein and queens
  - https://review.opendev.org/742624
Reporting needs to be handled later.
updates on ironic tempest plugin
- Any blocker bugs?
- https://review.opendev.org/#/c/727692/ tempest conf patch
  - need help on addressing ironic job?
- BaremetalMultitenancy Test is failing on ironic side. Iury is working on fix.
- Depends on patch is not working: https://review.opendev.org/#/c/723876
Do we want to backport
- Remove checking of Accept-Range header: https://review.opendev.org/#/c/738629/
  - for train & ussuri for tempest in rdo-side?
    - yes, once a new tempest tag is released
containers collect logs update
scenario manager refactoring updates
Non zuul run
- https://review.opendev.org/#/c/742867/

2020-07-17

Switch to new skip list
- new jobs using skip list
  - https://review.rdoproject.org/r/28513 job definition
  - Test drive - https://review.rdoproject.org/r/28543
- Moving to new skip list
  - https://review.opendev.org/#/c/726773/
Updates on Ironic tempest plugin integration
Deprecate validate-tempest
- problem : cloud.yaml needs to be generated for queens and stein
- container collect logs is too slow ?
  - kopecmartin: no updates yet, didn't have time - I was half of the week on PTO

2020-07-10

Pushing tempest skip list to production -> by monday
- Adding periodic jobs with new temepst skip list
  - List of plugins used:
    - neutron -> sc7
    - cinder -> sc2
    - octavia -> sc10
    - designate -> sc3
    - ironic -> sc12
- Finally deprecate validate-tempest role
  - Give a try to switch to os_tempest for queens and stein
  - before deprecating, we need to merge ironic ostempest
Remove multiple enterance of cirros image var used at multiple place
- Ideas welcome
- Defined at multiple places
- Review dib images for cirros for upstream, rdocloud, vexxhost and downstream, ostempest, default tempest
- Arx is working on that{need a user story for this}
cr repo patches reviews
- https://review.opendev.org/#/q/topic:crrepo+(status:open+OR+status:merged)
- https://review.rdoproject.org/r/#/q/topic:crrepo+(status:open+OR+status:merged)
- [Question] Since these repos does not exists, can we add a task just for checking url and add it with existing fs02 job
  - let it pass when repo will be available it will start failing, put it in promotion criteria?
  - weshay to pass the rlandy zuul additonal repo experiment.
Share what you are doing in tempest usptream
- tempest scenario manager
- tempest CLI unit test coverage
  - https://blueprints.launchpad.net/tempest/+spec/tempest-cli-unit-test-coverage
  - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_68/677268/1/check/openstack-tox-cover/34f43ba/cover/
container collect logs is too slow:
- https://github.com/openstack/ansible-role-collect-logs/blob/master/files/collect-container-logs.sh
- increased log collection time in downstream from ~6 to ~12 minutes
  - Need to compare with upstream and downstream logs

2020-06-19

Tracking victoria PTG tempest work in taiga
- Tempest workfile tests (@arxcruz
- https://etherpad.opendev.org/p/qa-victoria-priority
Need to setup fs021 based on centos-8 for train, ussuri and master
- https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-skipped-tempest-master
- with cr repos enabled
  - fs002 cr job for c8 train, ussuri & master
- with new temepst skip list
  - List of plugins used:
    - neutron -> sc7
    - cinder -> sc2
    - octavia -> sc10
    - designate -> sc3
    - ironic -> sc12
  - use periodic-tripleo-ci-centos-8-standalone-skipped-tempest-master as a parent and use skip_test as and run in periodic weekend
Dropping tempest-container from the doc: https://bugzilla.redhat.com/show_bug.cgi?id=1844084
- Do not drop the container
  - Think about v2 for containers without mounting directories
Updates on tempest skip list
*

2020-05-29

tempest container design
- https://tree.taiga.io/project/tripleo-ci-board/us/1776?milestone=266431
- Discuss about how user can consume tempest container to run tempest
  - Just don't use it
  - Tempest containers are built using kolla and here is the dir mounted https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/undercloud/tempest-container-puppet.yaml#L51
  - https://github.com/openstack/tripleo-heat-templates/blob/master/environments/enable_tempest.yaml

Urgent tempest bug

https://bugzilla.redhat.com/show_bug.cgi?id=1753643
- @arxcruz
https://bugzilla.redhat.com/show_bug.cgi?id=1835301
https://bugzilla.redhat.com/show_bug.cgi?id=1839468
- ACTION ITEMS
  - first fix the container
  - update the tempest / container execution procedure
  - put a branched tripleo jobs on python-tempestconf that use the tempest container
    - stein job voting
    - train job voting
    - ussuri job voting
    - master job voting
  - update / close bug
skip list tool update
- Does this job working https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-skipped-tempest-master/eafe356/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz ?
  - Not yet, I found a bug when it's running in periodic, it's easy to do a workaround, but I want to proper fix it.
- Regarding thinking the schema of skip test yaml file
  - Use big yaml file or putting tests based on components?
    - Big yaml file
      - would be good since we have jobs taking less time
  - We can present this during Tempest PTG
Octavia and Ironic tempest plugin update
*
Things to attend during PTG
- Tempest
  - openstack-tempest-skiplist
  - need to takes notes what is coming for victoria
    - Anything impacting TripleO
      - Like deprecations or removal
  - tempest scenario manager by sonia
    - PTG QA topics https://etherpad.opendev.org/p/qa-victoria-ptg
AI chandan
- openstack-tempest-skiplist documentation on docs.openstack.org
- Train temepst version bump

2020-05-15

ussuri skip list review
- https://review.opendev.org/#/c/728134/
- Arx to put all in a single yaml file
- brain storm
  - Based on components put the tests
  - Rethink about big yaml file
skip list tool updates
- Almost done, need jobs now exercising it
Octavia updates:
- [TQE] https://review.opendev.org/727739
- [TQ] https://review.opendev.org/726752
- [OSA] https://review.opendev.org/727949
- python-tempestconf patch coming
- Few other requirements from octavia team
  - https://tree.taiga.io/project/tripleo-ci-board/task/1748
    - Nest kvm based for testing full octavia
      - upstream nodepool are qemu based jobs
      - Do we need additional fs as a standalone triggered via third party
      - ovb job for octavia for component pipeline
      - reusing fs038 for multinode
Ironic updates:
- Python Tempestconf
  - https://review.opendev.org/#/c/727692/
- Openstack OS Tempest
  - https://review.opendev.org/#/c/720705/
  - Blocked:
Need to start working on tempest bump as c8 train jobs are available

2020-05-11

Tempest bump:
- Train: Problems blocking tempest-24.0.0
- Stein: https://review.rdoproject.org/r/#/c/26769/
  - what featuresets to run it against?
Jobs for testing skip list with openstack-tempest-skiplist
- https://tree.taiga.io/project/tripleo-ci-board/task/1659
- standalone is manageable
  - api and scenario as a source of truth
  - May be need a new env file to manage all scenario
    - run as a weekend pipeline standalone , sc1 -4, fs020
    - trigger any pipeline on demand
- skipped test needs to be triggered via zuul var
- use fs020 as a refrence for full-tempest
- featureset020 deprecated?
Ironic integration update
- waiting on ci
- https://review.opendev.org/#/c/723876/ on tripleoci
- need a job on OSA side to run ironic tempest side, make changes in openstack-ansible-os_ironic
Octavia integration update
- https://review.opendev.org/#/c/726752/
Bug triaging:
- https://bugzilla.redhat.com/show_bug.cgi?id=1825611
  - left without attention for some time
  - what's the status and who's gonna work on that?
  - Need to check with relase delivery on puppet-tempest
Cirros md5 check
- rocky, stein, queens
- cirros url updated and nodepool download the new url and got html page, but there is a no image, glance spin the image and
  nova set it active, need to find a way to check image is valid or not
  - need to check the image and md5 checksum
    - our side: cached image is wrong or same (make sure md5 matches) via library
      - Need to think before implementing as it is for older branches
    - glance or nova needs to check for the valid image [may be rfe]

2020-04-24

Update tempest version
- stein - tempest-23.0.0
- train - tempest-24.0.0
  - Need centos-8 tripleo CI jobs
  - chkumar to work with wes
Questions to be answered from common tempest role
- https://hackmd.io/LOk-8PzQQ1Wu5PwgKlBw3w#Questions–What-don%E2%80%99t-we-know-yet
Updates on skip list mvp
*

2020-04-17

Updates on Ironic tempest plugin
- Amol is rucking, who is taking care of his tasks?
  - Ok, hs is going to work parallely.
- What is left to mark it completion/help?
  - Add the playbook https://opendev.org/openstack/openstack-ansible-os_tempest
skip list management updates
- Things left for completing MVP
  - soniya taking care of generating command to skip list : https://review.opendev.org/#/c/720434/
- Need to have a proper flip switch for moving to new tool
Pre planning for os_tempest integration in infrared
Ussuri requirements sync for tempest plugins
- New tag for tempest for ussuri release
- rfolco is leading this one
Help on tempest bugs(upstream/downstream)
- need help on tempest container Urgent
  - https://bugzilla.redhat.com/show_bug.cgi?id=1753643
https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/undercloud/tempest-container-puppet.yaml#L52

2020-04-03

Updates on Ironic tempest plugin
- <wes> needs help w/ devel environment, please refer to https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8fd/716879/5/check/tripleo-ci-centos-8-scenario012-standalone/8fd9212/job-output.txt </wes>
skip list management updates
- repo up: https://opendev.org/openstack/openstack-tempest-skiplist
- schema review: https://opendev.org/openstack/openstack-tempest-skiplist/src/branch/master/doc/source/yaml/formatting.rst
- How it will be consumed with os_tempest? may be example will help
  - Needs various schema examples
    - jobs running on multiple branches
  - How it is going to be consumed with os_tempest or seperatly?

2020-03-27

Ironic tempest plugin integration
- https://hackmd.io/BWr-GGztSS6So-i5aWHBYA
skip list management review
- https://hackmd.io/twHmV2f2SGCMvfHjLMoGoA?both
- Best test case of laying implementation
  - https://launchpad.net/bugs/1869077
Updates on collect logs

2020-03-20

Skip list management proposal review:
- https://hackmd.io/twHmV2f2SGCMvfHjLMoGoA?both
Updates on collect_logs integration
- Infrared plugin installation improvement
  - https://review.gerrithub.io/c/redhat-openstack/infrared/+/486423
  - https://review.opendev.org/#/c/713325/
- Pavel is working on downstream transition - there's more challenges than expected

2020-03-13

Skip list management for integration and component pipeline as well as downstream
- https://hackmd.io/twHmV2f2SGCMvfHjLMoGoA?both
- Cases on handle
  - Skip tests based on
    - deployment target
    - jobtype
    - tempestconf flags
    - global skip
Updates on collect_logs integration
RDO cloud decommission
tempest container bug - https://bugzilla.redhat.com/show_bug.cgi?id=1753643

2020-03-06

Rethinking component pipeline tempest tests and skip list
- Run a set of common scenario tests in each job in component pipeline
- And component specific tests
- Skip list
  - common skip list go to global skip
  - component failing tests
    - At job level
    - in yaml file stating this tests failed at particular job.
Updates on collect_logs integration
- mkopec will be sending the emails

2020-02-21

Tempest Concurreny impacting gate jobs
- Add –worker-file option in tempest - https://review.opendev.org/#/c/708833/
  - How to consume with os_tempest and tripleo?
Downstream CI - how difficult would it be to add a new Jenkins job which would package the change (scratch build basically) and put a link to brew in a comment?
- Idea: Automation blocker wouldn't be a real blocker
- Goal: provide the newest tempest for automated jobs
  - the changes would be immediately verified and the automation could continue discovering other bugs
Updates on collect_logs integration
- https://review.opendev.org/#/c/703765/ Dynamically generate a list of files for rsync
- mkopec now working on decoupling collect and publish part
- fastforward storage efforts

2020-02-07

Updates on collect_logs integration

2020-01-31

Updates on collect_logs integration
- Infrared plugin patch merged now
- Any other updates?
- Working on ara refactoring
  - https://review.opendev.org/#/c/704762/
- GOAL: consume infrared in real downstream jenkins jobs
  - list of files needs to be collect by infrared
  - infrared uses find command
  - collect-logs uses rysnc
    - 90% files getting collected using collect-logs role using infrared
OSA collect logs integration
- It's kind of on hold for now, let's first work on the collect-logs + infrared stuff first, while moving slowly the osa + collect-logs
Tempest downstream tempest patchs in internal sf
- POC based on podman https://code.engineering.redhat.com/gerrit/#/c/190836/
  - short term rhpkg
  - longer term goal zuul-distro-jobs
Updates on tempest tag 23.0.0 train and neutron tempest plugin master
- https://review.rdoproject.org/r/#/c/24683/
  - full tempest standalone passing
  - fs020 timing out
    - I have a POC of a fs020 running with more cpu and 4 threads (concurrency) running in 4:30. However, I don't have the logs because of the post_failure errors.
    - I've notice there are free ram and cpu wasn't too high, so I'm testing today with concurrency 5
  - fs21 https://logserver.rdoproject.org/83/24683/8/check/distgit-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset021-train/2f743c1/logs/undercloud/var/log/tempest/stestr_results.html
    - 15 passing tests
- Tempest drop support to python 3.5, 23.0.0 is the last one, do we need to upgrade our python version?
  - RHOS-15 and RHOS-16, need to find python3 version

2020-01-24

Updates on collect_logs integration
- https://review.opendev.org/gitweb?p=openstack%2Fansible-role-collect-logs.git;a=commitdiff;h=330ac9eee90085553bdd3ac4c683d72dd09e1d6f
Updates on OSA integration collect-logs
- https://pkgs.org/ -> to find packages for different distro
- TODO: need to create a hackmd with the problems and plans
  - What about going by distro by distro
    - MVP start with debian
Updating tempest tag to 23.0.0 till queens?
- pre-req
  *need to finish skip list first
Skip list updates?

2020-01-17

Updates on collect_logs integration
- https://review.opendev.org/#/c/682042/ -> needs to merge it
- status report sent internally
- Next action item?
- Links in update email
  - 1] https://code.engineering.redhat.com/gerrit/#/c/182919/
    [2] https://github.com/kopecmartin/ansible-role-collect-logs
    [3] http://10.0.145.186/
- Files collected upstream: https://github.com/openstack/tripleo-ci/blob/master/toci-quickstart/config/collect-logs.yml

A link to the JIRA ticket for a reference: https://projects.engineering.redhat.com/browse/RHOSINFRA-2954

Tempest bugs affecting TripleO CI
Skip list remaining work items

2020-01-10

Updates on collect_logs integration
- https://tree.taiga.io/project/tripleo-ci-board/us/1265
- ready for merge: https://review.opendev.org/#/c/682042/
- scoping out for collaborative work?
  - https://projects.engineering.redhat.com/browse/RHOSINFRA-2954
Skip list remaining work
Tempest bugs needs attension
- https://bugs.launchpad.net/tripleo/+bug/1856723
  - https://bugs.launchpad.net/tripleo/+bug/1744907

2019-12-13

Updates on collect_logs integration
Updates on skip lists
- Cleaning up skip list
- Updates on the tooling

2019-11-29

os_tempest job porting done
Updates on collect-logs
- https://review.opendev.org/#/c/682042/ infrared plugin
  - How to tell reviewers it is tested?
    - May be push the logs to rdo logserver from infrared run?
      - https://code.engineering.redhat.com/gerrit/#/c/182919/
Skip list management review
- https://review.opendev.org/#/c/696387/

2019-11-22

os_tempest job porting
- fs027 got stuck due to cavets in exposing undercloud=true skip list
  - https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/playbooks/tempest.yml#L64
- porting to fs030 and replacing it with scenario7 standlaone and adding in promotion pipeline
- Need to deprecate fs030 multinode
Updates on collect-logs and tempest bug work
Skip list management
- What is missing to complete the project?
  - Proper representation of data (whatever dumped in mariadb database) within grafana

2019-11-15

Updates from PTG
- Needs reviews add it to tempest meeting wiki page
- Help in traiging Bug upstream and reviews
- Usseri release work https://etherpad.openstack.org/p/qa-ussuri-priority
  - Tempest cleanup in gate and check
  - Periodic in some job
    - May be in FS021?
- Octavia integration and collect_logs in OSA
Updates on os_tempest porting
- fs030 done
- fs039 in progress
  - Take a look standalone version
- fs027/fs030 needs help
Putting tempest tests with in scenario folder itself
- https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/ci/environments
Tempest Bug work
Skip list managements updates

2019-11-01

Updates on collect-logs work
- https://tree.taiga.io/project/tripleo-ci-board/us/1265
- [mkopec] Infrared plugin ready - https://review.opendev.org/#/c/682042/
- Tested in stage env
- Next goal: find out what is missing in our collect-logs from infrared side
  - He will be sending a followup email on monday to the team
Most of the multinode jobs moved to os_tempest
- Remaining
  - undercloud job
  - Need to work on support for octavia and Ironic coverage
No new work this sprint as the team is busy with other sprint works
PTG topics
- [QA] https://etherpad.openstack.org/p/shanghai-ptg-qa
  - Tempest Cleanup and Tempest volunteer for reviewing patches
- [OSA] https://etherpad.openstack.org/p/PVG-OSA-PTG
  - Updates on collect-logs and os_tempest
Updates from Skip list

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`	在筆記中貼入程式碼
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.

Tripleo Tempest Meeting

tags: Meeting

Links:

Ping list

Liaison: svyas

Agenda

2022-06-29

2022-06-7

2022-06-1

2022-05-18

2022-05-04

2022-04-13

2022-03-16

2022-03-09

2022-03-02

2022-02-09

2022-02-09

2022-01-19

2022-01-12

2022-01-05

2021-12-15

2021-12-01

Agenda

2021-11-24

Agenda

2021-11-10

2021-10-27

2021-10-12

2021-10-06

2021-09-29

2021-09-22

2021-09-15

2021-09-08

2021-09-01

2021-08-18

2021-08-11

2021-08-04

2021-07-07

2021-06-30

2021-06-23

2021-06-16

2021-06-09

2021-05-19

2021-05-12

2021-05-05

2021-04-29

2021-04-07

2021-03-31

2021-03-24

2021-03-10

2021-03-03

2021-02-24

2021-02-03

2021-01-27

2021-01-20

2021-01-13

2020-12-09

2020-11-25

2020-11-18

2020-11-11

2020-11-04

2020-10-23

2020-10-14

2020-10-07

2020-09-23

2020-09-16

2020-08-21

2020-08-07

2020-07-31

2020-07-24

2020-07-17

2020-07-10

2020-06-19

2020-05-29

Urgent tempest bug

2020-05-15

2020-05-11

2020-04-24

2020-04-17

2020-04-03

tags: `Meeting`