# Gluster Minor Release Tasks (for release owner(s))

A gluster minor release is a bug fix release, on a currently supported major release. The minor releases occur every month at pre-announced days of the month (see [1]).

Considering a month as roughly 30 days, the release tasks beow are listed as *Day #* sections.

## General responsibilities of a release owner
Release owner(s) are the only ones that have merge rights on release branches in gerrit, and hence are responsible for the content in the relase and also the release schedule.

1. All commits that are merged into an minor release should adhere to the backporting guidelines [2].
2. Minor releases should adhere to the release calendar and should be released on time
3. Exceptions to the release calendar can occur only when,
- There are blockers introduced between the last minor release and the current one
- For what consititues a blocker, see Appendix A
- There is a planned asynchrounous minor release, to address a blocker that was identified post the minor release
- For what consititues a blocker, see Appendix A

## Day 0/1: (release start)
- Ensure minor release bug tracker is open
- Example: https://bugzilla.redhat.com/show_bug.cgi?id=glusterfs-3.12.1
- Bug should have an alias (glusterfs-3.12.0 in the above example)
- Announce the next minor release with details (mail),
- Day of release (10th/20th/30th)
- Tracker bug, and reminder for community to add blockers to the same
- Dashboard link for community to follow open patches against the release

## Day 5/10/15/20/25/30: (every 5 days, release content checks)
- Gerrit work
- Check release dashboard and merge patches that are ready
- Check release dashboard and review patches that are ready, but lack reviews, to accelerate merges
- Comment on any patch that potentially has issues (like change-ID inconsistency, or does not fit the backport criteria [2])
- Raise on the mailing lists, if there are new features backported to the release branch, as this needs good justification prior to merging

## Day 10/20/30: (every 10 days, release health checks)
- Check fstat [3] for regression failures
- Check if similar failures occur in master, to get hints on if the test is already marked bad, or if something is failing elsewhere as well
- Report to the mailing lists tests that are failing regularly (as these could be regressions introduced into the branch)
- Raise flags if something is unique to the release branch
- Ensure current minor release is uptodate with older minor releases
- Why: If a patch that fixes a bug is part of an older minor release and is missing from the later minor release, incentive to upgrade and stay at the later version is diminished. IOW, it is expected that a later minor release contains all the fixes made in the prior minor releases.
- How to:
- git cherry deltas between branches
- Trigger a Glusto run on the minor release branch
- Why: Ensures that Glusto runs are clean as the release progresses and not throw surprises during the end of the release
- How to: **PENDING**
- Trigger performance/workload benchmark run on the minor release branch
- Why: Ensures that fixes are not causing a major performance degrade and further ensures that core use-cases are sanity checked
- How to: **PENDING**
- Ensure line coverage tests are not regressing in coverage
- Why: Adding code that is not covered by tests is a good way to cause regressions post release, this is hence a health check to ensure that coverage remains the same since the start of the release
- How to: **PENDING**
- Ensure Coverity runs are not regressing
- Why: Reasoning is the same as for line coverage
- How to: **PENDING**
- Update the community on release health, based on above parameters (mail)
- Update the community on health irrespective of weather there are issues or not

## Day 28: (tagging preparation)
- Prepare release notes
- Use: https://github.com/gluster/release-tools/blob/master/release_notes.sh
- Remind community of impending release (mail)
- Add a reminder that any blockers not marked yet as such, need to be done ASAP
- Add details on release health, pending patches, pending blockers without patches, any other release related work that needs attention

## Day 30: (tagging day)
NOTE: All activities are done in order
- Finalize and commit release notes
- Pay special attention to "Major Issues" section to note any major issues that are carried forward or are new, still persist in the release.
- Tag the release [4]
- Kick off the Jenkins job for packaging [5]
- Upload release notes to the doc site [6]
- Baseline release health checks
- How to: **PENDING**

## Day 30(+2 at max):
NOTE: All activities are done in order
- Once packages are available,
- Test install, upgrade [7]
- Respond to package maintainers on testing feedback
- Announce the release in the announce lists
- Example: http://lists.gluster.org/pipermail/announce/2017-August/000081.html
- Close bugs that were part of the release
- Get the bug list, example: git log --format=email v3.11.2...v3.11.3 | grep -i ^bug: | awk '{print $2}' | sort -u > /tmp/bug-list.txt
- Prune the bug list to exclude any bugs that are not meant to be closed yet!
- Example: https://bugzilla.redhat.com/buglist.cgi?quicksearch=789278%201193929&list_id=7834052
- Use https://github.com/gluster/release-tools/blob/master/close-bugs.sh to close the bugs, providing the required parameters and the bug list as above
- Close the tracking bug
- Open next minor release tracking bug (Day 0 starts for the next minor release, rinse and repeat!)

## Tools tha thelp release owners
- fstat
- gerrit dashboards
- glusto
- TBD...

### Appendix A
**Rough notes**
- A change introduced in the current release cycle **ONLY** causing,
- A crash
- Data corruption
- Upgrade/Install incompatability

1. Anything already a BLOCKER issue and present in the prior release, is not a BLOCKER for the current release. Problem exists, and solution arriving late or in a day or two does not change the schedule!
2. Other issues that were introduced in the current release cycle, should not cause interruptions to the usage of Gluster, and problem can be noted in the release-notes as a "Major Issue"
3. In the extreme case, something does slip by and is actually a BLOCKER, we can take in the fix and make an out of band (i.e not on the next release date for the minor release), minor release for the branch under consideration

## Links and references
[1] Gluster release schedule: https://www.gluster.org/release-schedule/
[2] See "GlusterFS minor release window" in page: https://gluster.readthedocs.io/en/latest/Contributors-Guide/GlusterFS-Release-process/
[3] Example fstat range check for failures: https://fstat.gluster.org/summary?start_date=2017-07-20&end_date=2017-08-30&branch=release-3.12
[4] Release tagging how to: https://hackmd.io/JwQwZgphBMCsAsBaAzAYxMR8wEYAciwA7MhIgEbnwAm1ycss0EQA#
[5] Jenkins packaging job: https://build.gluster.org/job/release-new/build?delay=0sec
- Example: https://build.gluster.org/job/release-new/208/parameters/
[6] Example release-notes PR: https://github.com/gluster/glusterdocs/pull/254/files
[7] Install testing strategy (ATM, can be applied to upgrades as well): https://hackmd.io/GYIwTADCDsDMCGBaArAUxAY0QFhBAbIgJwCMySIwJmAJvGMBvNEA#