owned this note   changed 5 years ago  
Published Linked with GitHub

Commmon Tempest Role Design

tags: Design

What

https://github.com/openstack/openstack-ansible-os_tempest

WHY

tripleo and openstack-ansible have successfully shared a common tempest role 1.5 years. Let's share more code vs. write our own.
This enable us to have more people checking the code, more functionalities, and work more with upstream community.

Specific goals:

  • Write less code, share more
  • Drink more beer
  • A common skip list would get more tempest errors written up as bugs and keep failing tempest tests out of CI runs.
  • Save time, energy and sanity

Design

Current examples of TripleO and OpenStack Ansible

Current infrared

The two teams already share python-tempestconf link

"test -e ~/.virtualenvs/.tempest_conf/bin/activate && source ~/.virtualenvs/.tempest_conf/bin/activate\n source ~/keystonerc\n /usr/bin/discover-tempest-config  --deployer-input ~/ir-tempest-deployer-input.conf  --debug -v --create   --image http://rhos-qe-mirror-tlv.usersys.redhat.com/images/cirros-0.4.0-x86_64-disk.img    identity.uri $OS_AUTH_URL identity.admin_password $OS_PASSWORD scenario.img_dir ~/tempest-dir/etc            identity.region regionOne    orchestration.stack_owner_role heat_stack_owner       --out ~/tempest-dir/etc/tempest.conf
  • question: are we using the same python-tempestconf?

infrared-plugin

  • should be roughly the equivalant of the common-logging infrared plugin
  • example link
  • Will openstack-ansible allow for an infrared plugin file in the role? ( we think so )
    • If we expose an infrared based job voting against os_tempest repo, they will allow it.

shared skip list

  • Shared tempest skip list

supported tempest plugins

openstack-ansible supported tempest plugins link

missing tempest plugin support and tempestconfig generation in tempestconf

How are tests removed from the skip list?

  • Featureset021
    • This job run only the failing tests on the skiplist
    • If a test start to pass on featureset021, that means, the test is 'fixed'
    • In the cockpit there's a session to check these, once the test pass a few times, we can remove it.

Design requirements

  • Core rights on skip list repo
  • os_tempest will work from train release onwards.
  • Jenkins can read results
  • Any polarion requirements?
  • Tempest whitebox plugin link
    • It can be enabled just by adding tempest_service_enable_whitebox in ostempest default vars.
  • What else??

Questions / What don't we know yet???

  • ADD YOUR THOUGHTS AND QUESTIONS AND CONCERNS HERE!!!
  • Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    Installation of tempest (rpm package vs git source code) is this included in this work or the role assumes tempest framework is already on the system ?
    • Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      Yes, it is included, you can install by git, pip, rpm, etc
  • Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    Does design allows to use Depends-On tempest framework changes / plugins, eg. Depends-On: change from whitebox tempest plugin in development change
    • Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      On zuul side, yes, in jenkins, maybe @B_hkIsryS2Sxpv62Mkw57Q can answer
  • Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    in IR all post-deployment steps for preparing the env are extraced in 'cloud-config' plugin (like external network and such) - does this overlap? (psedlak)
    • Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      All these are parsed setting facts that we passes to os_tempest role
  • Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    ir tempest does have support for custom commands also custom package injection (imo mostly used for plugins or component specific tests) (psedlak)
    • Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      If this is related to packages to be installed based on plugins, os_tempest have that, custom commands, it doesn't, and it can be implemented as well
  • ir tempest has slight support for (psedlak):
    • [q] repeated runs (used by upgrade/update jobs - pre, middle, post upgrade)
      • Image Not Showing Possible Reasons
        • The image file may be corrupted
        • The server hosting the image is unavailable
        • The image path is incorrect
        • The image format is not supported
        Learn More →
        We are using this on our upgrade jobs, basically we call the playbook twice, with different options IIRC
    • [q] separated test-sets (runs tempest once for set A, then for set B - generates separated result files etc, not sure how much that is used or for what though)
      • Image Not Showing Possible Reasons
        • The image file may be corrupted
        • The server hosting the image is unavailable
        • The image path is incorrect
        • The image format is not supported
        Learn More →
        We don't have this, but doesn't seems to be hard to do
    • [q] i would recommend that someone with deep understanding of os_tempest/design goes over infrared tempest plugin spec (and code) and highlights the main differences/featue inparity (could be on some meeting with ir folks to speed it up)
    • Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      This is possible to implement in os_tempest I guess, right now we don't have this support.
  • Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    possibly octario integration should be considered too

INTERESTED IN HELPING?

  • weshay
  • arxcruz
  • psedlak-just-parttime/consultation
  • Chandan Kumar
  • Amol
  • marios
  • soniya

Meeting notes 4/22/2020:

Select a repo