owned this note changed 5 years ago
Linked with GitHub

Upgrade test scenarios

File plugin

Resources present before upgrade

  1. FileRepository with a single version synced from a FileRemote.
    2. Half artifacts synced, half not? (test remote artifacts) (dalley)
  2. FileRemote with a URL pointing to a repository with content set that's different from the repository version in 1.
  3. FilePublication that is created from the repository version in 1.
  4. FileDistribution A that points to the publication in 3.
  5. FileRepository
  6. Test downloading downloaded + lazy content from repository

RPM plugin

Container plugin

Ansible plugin

Maven plugin

Brainstorming

  • from which version to which version?
    • from the latest x.y.z release of pulpcore and plugin being to tested to the master branch of pulpcore and master branch of plugin
  • which database tables should we test?
    • all the content types, all remote types, all exporters, ALL resources
    • each plugin team should decide it
  • we want to test:
    • migrations
    • REST API
  • have each plugin upgrade test be a separate job
  • stretch goal - test all katello plugins together
  • add a check to regular CI that ensures no new migrations are generated when makemigrations is run
  • https://pulp.plan.io/issues/6637 - the epic for improving functional test
    • Should we combine 6637 effort with the upgrade testing?
      • if we don't combine, we will end up with 2 sets of tests
      • need to handle cases where a feature is not present in version N-1 and added in version N
        • tests for each plugin are already specific to the version they sit within
  • as a first step in this giant effort we should simply test that we can install a previous version of pulpcore + plugin and then run migrations for master branch of pulpcore + plugin.

Ideas from June 24

  • Requirements:
    • Create a set of tests that are run before an upgrade for pulp_file
    • Create a set of unique fixtures for pulp_file tests involved during upgrade testing
    • Remove the use of orphan cleanup for tests that are used by post_upgrade tests
    • Create a set of post-upgrade tests that rely on data generated by pre-upgrade tests
Select a repo