Arvados

h1. Release Checklist

Pre-process:

# Create an issue for the release.

# Add each of the following steps (starting at step 1) as tasks with the step number in the subject

# Assign each task

# The current task goes into the "In Progress" column

# When the current task is finished, move it to resolved, and move the next task into "In Progress"

# Notify the assignee of the next task that it is ready to begin

Meta-process:

# Periodically review this documented process reflects our actual process & update it

# When steps are added/changed/rearranged/removed, be sure to update the TASKS file used by the Arvados release tool: https://dev.arvados.org/projects/arvados/repository/arvados-dev/revisions/main/show/cmd/art

|_.Step|_.Who|_.What|

|0|engineering|Write lots of great code, build new features|

|1|engineering|Prepare release branch on @arvados@, @arvados-workbench2@ and @arvados-formula@.  For major releases this means branching a new @X.Y-staging@ from main.  For minor releases, this means cherry-picking features onto the existing @X.Y-staging@ branch.  Ensure that redmine issues for features or bugfixes that are appearing for the first time in this version are associated with the correct release (for major releases, use @art redmine issues find-and-associate@).|

|2|release eng|Update the "Upgrading Arvados and Release notes" doc page on @X.Y-staging@ branch with the version and date of the release.

On the @X.Y-staging@ branch, update these files to refer to the release version instead of @main@: 

* @arvados/tools/salt-install/provision.sh@ (set variables RELEASE="production" and VERSION="X.Y.Z-1")

* @arvados/tools/arvbox/bin/arvbox@ (update DEFAULT_TAG=X.Y.Z)

* @arvados/sdk/R/DESCRIPTION@ update "Version"

* @arvados/doc/install/arvbox.html.textile.liquid@ (update "refs/heads/main" to "refs/heads/X.Y-release")

* @arvados-formula/arvados/defaults.yaml@ (set variable (version: 'X.Y')|

|3|engineering|Ensure that the release staging branch passes automated tests on jenkins.|

|4|engineering|Review release branch, make sure all commits that need to be in the release are in the release, make sure that dependencies of Python and Ruby packages have upper version pins (to ensure that 3rd party dependencies updated post-stable-release don't break our code).  If new commits are added, resume checklist from step 3.|

|5|product mgr|Write release notes and publish them to www-dev site: https://www-dev.arvados.org/releases/|

|6|everyone|Review release notes|

|7|product mgr|Create a redmine release for the next patch release after the current one.|

|8|release eng|Build release candidate packages with version "X.Y.Z~rcN-1" using the jenkins jobs "build-and-publish-rc-packages":https://ci.arvados.org/view/Release%20Pipeline/job/build-and-publish-rc-packages/ and "workbench2-build-release-candidate-package":https://ci.arvados.org/view/Release%20Pipeline/job/workbench2-build-release-candidate-package/ .  Add a comment on the release ticket with the "arvados" and "arvados-workbench2" git commits that were used, as well as links to the jenkins runs.|

|9|release eng|Publish release candidate arvados/jobs Docker image using "docker-jobs-image-release":https://ci.arvados.org/view/Release%20Pipeline/job/docker-jobs-image-release/|

|10|ops|Test installer formula / provision scripts with RC packages - [[Installer development process]] ^1^|

|11|ops|"Build a new compute image":https://ci.arvados.org/view/All/job/packer-build-compute-image/ and deploy RC packages & new image to playground  ^1^|

|12|bfx|Run "bam-to-vcf pipeline":https://dev.arvados.org/issues/17049#note-7 on playground ^1^|

|13|engineering|Review release notes and https://dev.arvados.org/projects/arvados/wiki/Manual_testing_plan and perform final manual testing based on risk assessment.|

|14|product mgr|Approve RC for release|

|15|release eng|Build final release packages with version "X.Y.Z-1" using the jenkins jobs "build-and-publish-rc-packages":https://ci.arvados.org/view/Release%20Pipeline/job/build-and-publish-rc-packages/ and "workbench2-build-release-candidate-package":https://ci.arvados.org/view/Release%20Pipeline/job/workbench2-build-release-candidate-package/ .  Add a comment on the release ticket with the "arvados" and "arvados-workbench2" git commits that were used, as well as links to the jenkins runs.|

|16|release eng|Publish stable release arvados/jobs docker image using "docker-jobs-image-release":https://ci.arvados.org/view/Release%20Pipeline/job/docker-jobs-image-release/|

|17|release eng|Push packages to stable repos using "publish-packages-to-stable-repo":https://ci.arvados.org/view/Release%20Pipeline/job/publish-packages-to-stable-repo/ (see also https://dev.arvados.org/projects/ops/wiki/Promoting_Packages_to_Stable)|

|18|release eng|Publish Python packages and Ruby gems using "build-publish-packages-python-ruby":https://ci.arvados.org/view/Release%20Pipeline/job/build-publish-packages-python-ruby/|

|19|release eng|Publish Java package using "build-java-sdk":https://ci.arvados.org/view/Release%20Pipeline/job/build-java-sdk/ and following [[Releasing Java SDK packages]]|

|20|release eng|Publish R package using ???|

|21|release eng|Publish arvados/arvbox-demo docker image using "build-and-release-arvbox-image":https://ci.arvados.org/view/Release%20Pipeline/job/build-and-release-arvbox-image/|

|22|release eng|Tag the commits in each repo used to build the release in git. Create an annotated tag (@git tag --annotate@), with a message like: "Release X.Y.Z, release notes at https://arvados.org/release-notes/X.Y.Z/". That will make https://github.com/arvados/arvados/releases look really good. See "here":https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes for documentation about how to automate these.

Create or fast forward the @2.4-release@ branch to match @2.4-staging@.

Cherry-pick the upgrade notes commit (from step 2) on to @main@.|

|23|release eng|Ensure new release is published on https://doc.arvados.org ; ensure that release notes & any other materials are pointing to correct version of the docs.|

|24|product mgr|Merge release notes (step 6) from "develop" branch to "main" branch of @arvados-www@ git repository and check that https://arvados.org page is updated|

|25|product mgr|"Send out email notifications":https://docs.google.com/spreadsheets/d/18uaUPQ1r2_sQH18JTiblkNFcq8gsFflRAGm8-wvtQi4/edit?usp=sharing , tweet from the Arvados account, announce on the Discourse forum, gitter, etc|

|26|release eng|For major releases: Duplicate the jenkins configuration for the "run-tests" jobs tests to versioned names (run-tests-X.Y) so that future point releases remain testable after @main@ moves on|

|27|release eng|Add the release to "doi:10.5281/zenodo.6382942":https://doi.org/10.5281/zenodo.6382942

[[Updating Zenodo Version of Arvados after Release]]

https://zenodo.org/record/6382943|

^1^ If issues are discovered requiring additional commits, increase the release candidate version by one, and and resume the checklist from step 3.