---
# System prepended metadata

title: Release CI Improvements

---

# Release CI Improvements

## Goal:
* gather issues noticing during releases
* prioritize what to do first
* differentiate between "fix/first/fast" [F] and "change-proposal/soon" [RFE]

## Meeting Template
```
## MMM DD
### Previously (copy from prev-mtg-next-steps)
### Notes
### Next Steps
* post minutes to [Discourse thread](https://discourse.pulpproject.org/t/discussion-pulp-release-process-improvements/860/2)
```
## June 29

* Wrapping up working group
    * Last two action items
        * Automating update CI branches for Y releases
        * Automating Pulp-OCI-images release steps

## June 22
* Review PLEASE:
    * https://github.com/pulp/plugin_template/pull/751 [DONE]
* Last two items removing manual steps:
    * Remove need to manually update CI branches for Y releases [AI: gerrod]
        * need to have a programatically-reliable location of “list of supported branches”
        * change “…and latest-Y to just-released Y”
    * Automating Pulp-oci-images release steps
* What next?
    * working-group is set to end at end of June
    * we have only the auto-ci-branches Thing to do
    * looks like we can declare working-group Done at that point?
    * pulp-oci-images work should be done as well
        * discussion around pulp-oci-image status
        * work w/ decko on our needs here?
* Some proposed tasks (in proposed order):
    * get p-o-i to always have a branch for each pulpcore-y-branch
        * https://github.com/pulp/pulp-oci-images/issues/503 [dkliban]
    * pull apart p-o-i action so that the individual Things are separate actions
        * https://github.com/pulp/pulp-oci-images/issues/504
    * automate the p-o-i branch-creation in a way that makes it possible to invoke from pulpcore-release-workflow
        * https://github.com/pulp/pulp-oci-images/issues/505
    * automate p-o-i release in a way that it can also be invoked from core-release
* Action Items:
    * Remove need to manually update CI branches for Y releases [AI: gerrod]
    * https://github.com/pulp/pulp-oci-images/issues/503 [dkliban]

## Jun 15
* try to reserve brain-space to pick up items below as we can this week

    * [F] Fix conflicting auto-on-release PRs: Update Changelog vs Cherrypick Changelog
        * Possible fix - remove the cherrypick changelog, on bump minor version PR delete the changelog entries, solely have the update changelog job
        * https://github.com/pulp/plugin_template/issues/754
    * [F] Remove Tests from Release process
        * https://github.com/pulp/plugin_template/pull/751
    * [F] create new backport-label for Y-releases
        * https://github.com/pulp/plugin_template/pull/745
        * Should be converted to a separate workflow that only runs when plugin_template is modified
            * Should update the backport labels to reflect ci_update_branches
    * [F] Check if this job is necessary:
        * https://github.com/pulp/pulp-ci/actions/workflows/supported.yml
            * It updates a file in pulp-ci that is used in the docs header to tell if a user is looking at the latest version
            * Should change the logic to have the header look at that plugin's repository and then remove this job
            * Also the supported versions docs header should probably list the ci_update_branches

## Jun 8
* Removed tests from release workflow
    * https://github.com/pulp/plugin_template/pull/751
    * and there was Much Rejoicing!
    * eyes/review please
* Simplify (separate out) OCI release automation
    * All of the CI seems to be in one workflow file...
        * https://github.com/pulp/pulp-oci-images/blob/latest/.github/workflows/pulp_images.yml
        * From what I can tell this workflow is called in three scenarios: 
            * on pull-requests
            * daily cron-job
            * from api-request (usually called by the daily cron-job)
        * The logic of what the workflow does is dependent on how it is called (99% sure it is doing lots of redundant work, even within a single call)
    * Pulpcore <3.23 branches have their own special logic and use a different workflow file...
        * https://github.com/pulp/pulp-oci-images/blob/latest/.ci/scripts/update_ci_branches.py#L25-L28
        * https://github.com/pulp/pulp-oci-images/blob/3.21/.github/workflows/publish_images.yaml
            * Interestingly this special workflow isn't on the 3.22 branch, so the special logic check is wrong...
    * There might be a bug somewhere for the logic to release & tag updated Z-streams to not work if the corresponding Y image was never released
        * I think this is what happened to 3.25
* Some proposed tasks:
    * pull apart p-o-i action so that the individual Things are separate actions
    * get p-o-i to always have a branch for each pulpcore-y-branch
    * automate the p-o-i branch-creation in a way that makes it possible to invoke from pulpcore-release-workflow
    * automate p-o-i release in a way that it can also be invoked from core-release
* Is there a technical issue that gets in the way of separating our one-large-workflow into more tightly-focused files?
    * possibly? needs some investigation
## Jun 01
### Previously (copy from prev-mtg-next-steps)
* ~~focus on core/3.25-compat releases **first**~~
    * when does the oci-image get published?
    * not out yet - AI: decko - hit up Mike to figure out what's up
* try to reserve brain-space to pick up items below as we can this week
  * [F] Fix conflicting auto-on-release PRs: Update Changelog vs Cherrypick Changelog
  * [F] Remove Tests from Release process
  * [F] create new backport-label for Y-releases
      * https://github.com/pulp/plugin_template/pull/745
  * [F] Check if this job is necessary:
    * https://github.com/pulp/pulp-ci/actions/workflows/supported.yml
  * [F] disable deprecations-test from release-branches "automagically"
    * yes please
### Notes
* Check release script:
    * https://github.com/pulp/plugin_template/pull/750
    * looks thru CHANGES directory, does Cool Stuff in response
    * Note: we don't appear to currently have git-python as a requirement anywhere
        * is there a "release_requirements.txt" somewhere? should there be?
            * release_requirements.txt is under a DEPRECATED_FILES in `plugin_template`
        * ci_requirements? (no - stuff that needs to be installed before tests, not release)
        * we should separate this-script, from "we should have a releasae_requirements.txt file" - that will be its own PR
*  gerrod is picking up "remove tests from release" ASAP 
    *  backport-releases took up A LOT of time
    *  the one-off/transient test-failures are A Pain
* high-level goal: document what the CI is "actually doing", so ppl have a place to go to try and figure out problems
    *  a to-do for "after everything else is stable/quiescent"
### Next Steps
* AI: decko - hit up Mike to figure out what's up w/ oci-images and 3.25
* try to reserve brain-space to pick up items below as we can this week
  * [F] Fix conflicting auto-on-release PRs: Update Changelog vs Cherrypick Changelog
  * [F] Remove Tests from Release process
  * [F] create new backport-label for Y-releases
      * https://github.com/pulp/plugin_template/pull/745
  * [F] Check if this job is necessary:
    * https://github.com/pulp/pulp-ci/actions/workflows/supported.yml
  * [F] disable deprecations-test from release-branches "automagically"
    * yes please
* post minutes to [Discourse thread](https://discourse.pulpproject.org/t/discussion-pulp-release-process-improvements/860/2)

## May 25

### Previously
* "just do" things labelled [F] below
  * [F] Fix conflicting auto-on-release PRs: Update Changelog vs Cherrypick Changelog
  * [F] Remove Tests from Release process
  * [F] create new backport-label for Y-releases
      * https://github.com/pulp/plugin_template/pull/745
  * [F] Check if this job is necessary:
    * https://github.com/pulp/pulp-ci/actions/workflows/supported.yml
* ~~bring list to larger team for review/thoughts/observations~~
    * ~~https://discourse.pulpproject.org/t/discussion-pulp-release-process-improvements/860/3~~
* meeting weekly (Thurs 0900 GMT-5)
### Notes
* Finish releasing plugin pulpcore>=3.25 releases
    * file/rpm/python/maven/deb/certguard are done! /cheers wildly
    * container, ostree, ansible (actually release)
        * talk w/ lubos
        * AI [gerrod] ansible
    * cookbook, gem, npm for nightly (main branch only)
        * AI [decko] cookbook 
            * to be removed -> https://github.com/pulp/pulp-oci-images/pull/486
        * AI [gerrod x9c4] gem, ansible
        * AI [x9c4] npm
            * potentially remove from images for now
* [F] disable deprecations-test from release-branches "automagically"
    * yes please
* Working on check-release script (gerrod)
    * Multiple branches are missing changelog updates
        * seems to have been failures in changlog/remove-changefiles/update-dev?
        * needs investigation
        * seems not to affect any branches "we care about"
* discussion around "auto actions at create-release-branch action (Y branches)"
    * create new backport label (PR#745)
        * should prob take over from Michal since he's in finals for a few weeks
    * update ci-branches automatically
    * re-sync backport labels to to ci-branches
### Next Steps
* focus on core/3.25-compat releases **first**
* try to reserve brain-space to pick up items below as we can this week
  * [F] Fix conflicting auto-on-release PRs: Update Changelog vs Cherrypick Changelog
  * [F] Remove Tests from Release process
  * [F] create new backport-label for Y-releases
      * https://github.com/pulp/plugin_template/pull/745
  * [F] Check if this job is necessary:
    * https://github.com/pulp/pulp-ci/actions/workflows/supported.yml
  * [F] disable deprecations-test from release-branches "automagically"
    * yes please
* post minutes to [Discourse thread](https://discourse.pulpproject.org/t/discussion-pulp-release-process-improvements/860/2)

## May 18

### Next Steps
* "just do" things labelled [F] below
* bring list to larger team for review/thoughts/observations
    * AI: [ggainey] open a discourse thread
    * put on agenda for Monday's team mtg
* meeting weekly (Thurs 0900 GMT-5)

### Issues

* [F] Fix conflicting auto-on-release PRs: Update Changelog vs Cherrypick Changelog
    * why are there two?
    * which one is doing The Right Thing?
    * remove at least one
    * maybe it's a collision between Y-release/Z-release processes? 
* [F] Remove Tests from Release process
    * yes please
* Remove manual CI steps for OCI-images
    * https://github.com/pulp/pulp-oci-images/blob/latest/docs/developer-instructions.md
    * should be automatable in pulp-bot
    * can use args on the manual steps
    * or we can use args to programaticaly build the images
* Remove need to manually update CI branches for Y releases
    * need to have a programatically-reliable location of "list of supported branches"
    * change "...and latest-Y to just-released Y"
* [F] create new backport-label for Y-releases
    * [RFE] and remove unneeded ones?
    * see "reliable location of list of supported branches"
* [F] Check if this job is necessary:
    * https://github.com/pulp/pulp-ci/actions/workflows/supported.yml
    * is this "latest docs don't warn, prev docs say 'not most recent version'"?
    * The magic is happening here? https://github.com/pulp/pulp-ci/actions/runs/5010048693/jobs/8979570319#step:6:1
* Create script to determine if releases are needed
    * Checks the release branches for presences of changelog additions
    * can determine Y- vs Z- based on CHANGES extension
    * x9c4 has a git-log cmd
    * "some day", we could get to a completely-automated process
* Create script to auto-generate an announcement message
    * autopost to discourse?
    * autopost to twitter?
        * or mastodon?
        * y-release only? or all?
    * matrix channel $TOPICs?
    * slack too?