Arvados

h1. Release Checklist

Pre-process:

# Create an issue for the release.

# Add each of the following steps (starting at step 1) as tasks with the step number in the subject

# Assign each task

# The current task goes into the "In Progress" column

# When the current task is finished, move it to resolved, and move the next task into "In Progress"

# Notify the assignee of the next task that it is ready to begin

Meta-process:

# Periodically review this documented process reflects our actual process & update it

# When steps are added/changed/rearranged/removed, be sure to update "@cmd/art/TASKS@ in the @arvados-dev@ repository":https://dev.arvados.org/projects/arvados/repository/arvados-dev/revisions/main/show/cmd/art.

|_.Step|_.Who|_.What|

|0|engineering|Build new features, refine good code into great code|

|1|ops|"Build a new tordo compute image":https://ci.arvados.org/view/All/job/packer-build-compute-image/ against the latest development packages.

Test it with a couple representative workflows (at least one bioinformatics workflow and one S3 download workflow).

If everything works well, update version pins based on the versions installed in the new image. Update:

* source:tools/compute-images/ansible/roles/compute_docker/files/arvados-docker.pref

* source:tools/compute-images/ansible/roles/compute_nvidia/files/arvados-nvidia.pref

|

|2|engineering|Prepare release branch on the @arvados@ and @arvados-formula@ repositories.  For major releases, this means branching a new @X.Y-staging@ from main.  For minor releases, this means cherry-picking features onto the existing @X.Y-staging@ branch.  Ensure that Redmine issues for features or bugfixes that are appearing for the first time in this version are associated with the correct release (for major releases, use @art redmine issues find-and-associate@).|

|3|release eng|Update the "Upgrading Arvados and Release notes" doc page on @X.Y-staging@ branch with the version and date of the release.

On the @X.Y-staging@ branch, update these files to refer to the release version instead of @main@: 

* In @arvados/tools/arvbox/bin/arvbox@, update @DEFAULT_TAG=X.Y.Z@

* In @arvados/sdk/R/DESCRIPTION@,  update @Version:@|

|4|engineering|Ensure that the release staging branch passes automated tests on Jenkins.

For major releases, this means:

* "developer-run-tests":https://ci.arvados.org/job/developer-run-tests/

* "developer-run-tests-doc-sdk-java-R":https://ci.arvados.org/job/developer-run-tests-doc-sdk-java-R/

For minor releases, this means:

* X.Y-developer-run-tests

* X.Y-developer-run-tests-doc-sdk-java-R|

|5|engineering|Review release branch to make sure all commits that need to be in the release are in the release.  If new commits are added, resume checklist from step 3.|

|6|product mgr|Write release notes and publish them to www-dev site: https://www-dev.arvados.org/releases/|

|7|everyone|Review release notes|

|8|product mgr|Create a Redmine release for the next patch release after the current one.|

|9|release eng|Build release candidate packages with version "X.Y.Z~rcN-1" using the Jenkins job "build-and-publish-rc-packages":https://ci.arvados.org/job/build-and-publish-rc-packages/.  Add a comment on the release ticket identifying the Git commit hash used for the build, and link to your Jenkins run.|

|10|release eng|Publish release candidate @arvados/jobs@ Docker image using "docker-jobs-image-release":https://ci.arvados.org/job/docker-jobs-image-release/|

|11|ops|Test installer formula / provision scripts with RC packages - [[Installer development process]]|

|12|ops|"Build a new compute image":https://ci.arvados.org/job/packer-build-compute-image/ and update Playground using the new RC packages and compute node image ^1^|

|13|bfx|Run "CWL conformance tests":https://ci.arvados.org/view/CWL/job/arvados-cwl-conformance-tests/ on jenkins and "bam-to-vcf pipeline":https://workbench.pirca.arvadosapi.com/workflows/pirca-7fd4e-ut5n6r2ydl6o6kj on Playground|

|14|engineering|Perform final manual testing based on risk assessment, the release notes and "manual testing plan":https://dev.arvados.org/projects/arvados/wiki/Manual_testing_plan.|

|15|product mgr|Approve RC for release|

|16|release eng|Build final release packages with version "X.Y.Z-1" using the Jenkins job "build-and-publish-rc-packages":https://ci.arvados.org/job/build-and-publish-rc-packages/. Add a comment on the release ticket identifying the Git commit hash used for the build, and link to your Jenkins run.|

|17|release eng|Publish stable release @arvados/jobs@ Docker image using "docker-jobs-image-release":https://ci.arvados.org/job/docker-jobs-image-release/|

|18|release eng|Push packages to stable repos using "publish-packages-to-stable-repo":https://ci.arvados.org/job/publish-packages-to-stable-repo/ (see also https://dev.arvados.org/projects/ops/wiki/Promoting_Packages_to_Stable)|

|19|release eng|Publish Python packages and Ruby gems using "build-publish-packages-python-ruby":https://ci.arvados.org/job/build-publish-packages-python-ruby/|

|20|release eng|Publish Java package using "build-java-sdk":https://ci.arvados.org/job/build-java-sdk/ and following [[Releasing Java SDK packages]]|

|21|release eng|Publish R package using "build-package-r":https://ci.arvados.org/job/build-package-r/|

|22|release eng|Publish @arvados/arvbox-demo@ Docker image using "build-and-release-arvbox-demo-image":https://ci.arvados.org/job/build-and-release-docker-arvbox-demo-image/|

|23|release eng|Tag the commits in each repo used to build the release in Git. Create an annotated tag (@git tag --annotate@) with a message like "Release X.Y.Z, release notes at https://arvados.org/release-notes/X.Y.Z/" That makes the "GitHub releases page":https://github.com/arvados/arvados/releases look good. See "GitHub documentation for more details about how to automate releases":https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes.

Create or fast forward the @X.Y-release@ branch to match @X.Y-staging@.

Cherry-pick the upgrade notes commit (from step 2) onto @main@.|

|24|release eng|Ensure new release is published on https://doc.arvados.org/.

Ensure that release notes & any other materials are pointing to correct version of the docs.|

|25|ops|Update pirca and jutro to stable release|

|26|product mgr|Merge release notes (step 6) from "develop" branch to "main" branch of the @arvados-www@ Git repository and check that the https://arvados.org front page is updated|

|27|product mgr|Send out the release notes via MailChimp, tweet from the Arvados account, announce on the Discourse forum, Matrix, etc.|

|28|release eng|For major releases: Duplicate the Jenkins configuration for the "run-tests" jobs tests to versioned names (run-tests-X.Y) so that future point releases remain testable after @main@ moves on.  Note: if a Jenkins job isn't runnable after being copied (there no run button, or the multijob prints NullPointerException trying to run it), try disabling and re-enabling it.|

|29|release eng|Add the release to "doi:10.5281/zenodo.6382942":https://doi.org/10.5281/zenodo.6382942

[[Updating Zenodo Version of Arvados after Release]]

https://zenodo.org/record/6382943|