# Compose-id pinning and promotion workflow ## Brainstorming 1. Compose-id pinning: * 2 labels so far: * centos-ci-testing (periodic promotion of CentOS latest-compose) * current-centos (latest known good compose-id) 2. Compose-id promotion workflow: * latest-compose -> centos-ci-testing * Promoter code work already started in ci-config * Do we need more labels here? e.g: previous-centos-ci-testing * centos-ci-testing -> current-centos * needs a nodepool image pinned to centos-ci-testing compose-id * we can use an image pinned to an older compose, and then update repos + update packages * needs to define jobs and promotion criteria * will need to promote the nodepool image to current-centos * We can use an images pinned to an older compose-id and update repos with an specific compose-id. Only compose-ids will need to be promoted. ## TripleO Repos and Centos Stream Compose Pinning Date: Nov 11, 2021 Current Situation on CS9: * CS9 nodepool image has centos.repo which pulls packages from centos stream mirrors. * Pre-installed packages on nodepool images are from centos stream mirrors. * In tripleo CI jobs, we enable compose repos using tripleo-repos or using release file to lay down to compose repos. * Here we mix packages and metadata from centos mirror and compose * Which further leads to following issues * https://bugs.launchpad.net/tripleo/+bug/1950466 - CS9 nothing provides libgcc >= 11.2.1-6.el9 * Enable compose repos in pre: https://review.opendev.org/c/openstack/tripleo-ci/+/817455 * https://bugs.launchpad.net/tripleo/+bug/1949601 - package buildah filtered out by modular filtering * Use module_hotfix: https://review.opendev.org/c/openstack/tripleo-quickstart/+/816648 Long term Plan: * In TripleO CI integration and upstream check/gate jobs, we need to pull centos contents from centos mirrors. * Contents gets synced from centos mirror to opendev mirror (https://review.opendev.org/c/opendev/system-config/+/817136) * Once CentOS Stream 9 gets released. Main Goal: * Catch these issues early in the pipeline. * One Idea: Can we use CentOS Stream Pinning? * Life span of a compose content is approx 20 days * Latest compose will be generally pushed to mirrors (Need to clarify)? * Two composes (which one to use) * production: https://odcs.stream.centos.org/production/ * Development: https://odcs.stream.centos.org/development/ * Current Solution (We are driving) * Use tripleo-repos to enable compose in periodic line * Switch to mirror once contents gets synced Notes: * First: get system config patch in and use mirror content everywhere * switch to using the mirror content and no longer composes in release files * retest container and images everything * Second: Get the dependency line * Understand centos release pattern * mirror content is n -1 on prod composes if true - forward looking n with latest-compose else latest-compose is n-1 and then test with development compose * Store old compose ids * Build nodepool images with a particular compose ## Tasks ### CentOS Composes - Dependency Pipeline 1. Add a new repo type, 'repo_composes', to 'repo-setup' role that configures repos based on a compose-id * we can easily set the compose-url in the release files and use yum-config * WIP: https://review.opendev.org/c/openstack/tripleo-quickstart/+/819019 2. Create a new dependency repo_config for compose-pinning * we may want to overwrite 'repo' and 'add_repos' vars to avoid compose-repos being overwritten * make sure that we have delorean repos and deps repos being set 3. Implement function to get compose-url from humam-readable label * today the content is a compose-id, but we can change content to compose-url (easier to handle between different distros - which may have different urls) * e.g. https://images.rdoproject.org/centos_compose/centos8/master/centos-ci-testing (compose-id) 4. Create a new or update an existing 'pre' to use tripleo-yum-config to generate compose repos * it shall configure repos and update packages * not all the jobs use release file content to setup repos * the 'pre' should validate if the current image is too new for the compose-if being tested * where this info leaves in image? 4. Create new jobs * list of jobs TBD * first, validate with a provider + scenario * define image to be used for compose-pinning, need to be a old compose-id 6. Create pipeline and add to rdo-jobs * run every day or two? 7. Generate diff repo list * Use/adjust existing playbooks to compile the diff repo list ### CentOS Composes - Promotions 1. Create promotion jobs: * promote latest-compose-id to centos-ci-testing * periodic-tripleo-centos-8-promote-latest-compose-master * need to add to periodics * https://review.rdoproject.org/r/q/topic:%22compose-promoter%22+(status:open%20OR%20status:merged) * Update to include previous if needed (ci-config) * promote centos-ci-testing to current-centos * Improve promoter code to also promote based on criterias * see https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34612 * define criteria * create promotion environment file ## Questions 1. Which image should we use in pinning jobs? 2. How do we check which compose-id was the current in image creation?