Arvados

Release branches:

• Arvados branch 2.3-dev at 63a6e6e
• WB2 branch 2.3-dev at arvados-workbench2|e5198f5 (just branched off main with no additional changes)

2.3-dev branches:

• Arvados repo: 42bbacc
  • Test run: developer-run-tests: #2709
  • WB1 integration tests re-run: developer-run-tests-apps-workbench-integration: #2877
• Workbench2 repo: arvados-workbench2|a5f5a2e
  • developer-tests-workbench2: #492

2.3.0~rc1-1 RC Package builds:

• build-and-publish-rc-packages: #105
• workbench2-build-release-candidate-package: #73

2.3.0 Release notes¶

The Arvados team is pleased to announce Arvados 2.3.0. This is a significant upgrade, with many new features as well as bug fixes. We recommend that new and existing installations of 2.2.2 or earlier upgrade to 2.3.0. See Upgrading Arvados for upgrade instructions.

Major Features¶

• Basic LSF Support: (#17756, #17857)
  • Containers can now be dispatched to an LSF cluster, with some limitations (https://doc.arvados.org/main/install/crunch2-lsf/install-dispatch.html)
• Basic Singularity Support (#17296, #17813, #17816, #17755, #17857, #17726, #18098)
  • Arvados can run containers using Singularity instead of Docker, with some limitations (https://doc.arvados.org/main/install/singularity.html)
• Complete Storage Classes Support
  • Enhanced support in Keepstores, SDKs, client tools & Workbench2 allows uploading data directly to specific storage classes instead of relying on Keepbalance to eventually move the data to its final place. (#17351, #17388, #17389, #17392, #13382, #17394, #17393, #17465, #11184, #17572, #17573, #17574, #17573, #17928)
  • Ability to set storage classes for intermediate & final outputs (#17390, #17395)
  • Exported cluster configuration publishes the cluster's default storage class(es) (#17696)
  • Parallelize writes inside keepstore on multiple storage classes (#17698)
  • Prioritize reads from different storage classes (#17967)
  • Support storage classes filtering on collections (#17994, #17995)

Improvements¶

• WB2 integrates the webshell (#17229)
• WB2's new collection file browser allows loading and navigating through more significant collections (#17585)
• Group syncing tool allows case insensitive username matching (#18097)
• Collection manifest signing performance (#18051, #17217)
• Removed deprecated FTS database indexes, allowing collections with lots of files/directories (#15430)

Bug fixes¶

• WB2 pagination display fix (#18029)
• WB2 correctly shows the upload speed (#16655)
• WebDAV caching fix improves performance on huge collections (#18051)
• PySDK honors user-defined timeout (#17989)
• PySDK FUSE driver locking fix further increases its robustness (#18078)
• Cached user records handling in a satellite cluster fix auto-resolves username collisions (#18076)
• Removed unused update_uuid endpoint (#18094)
• Request-ID propagation fixes improve error traceability and debugging (#17830)
• Keepproxy spurious 413 error fix avoids future debugging confusion (#16665)

Release notes draft committed to https://www-dev.arvados.org/release-notes/2.3.0/

Lucas Di Pentima wrote:

Release notes draft committed to https://www-dev.arvados.org/release-notes/2.3.0/

Release notes reformatted using the same style as other major releases.

This new release adds limited support for dispatching containers to IBM LSF clusters, with some restrictions, allowing the integration of Arvados to additional enterprise environments.

I think it's called "Spectrum LSF" to distinguish from earlier versions, as far as I can tell the name "Spectrum" is just branding but it's good to be specific.

Remove the word "limited" and the phrase "with some restrictions" unless you are going to say what those are.

I would also change "enterprise" to "High Performance Computing" or just "HPC".

In addition to LSF support, this major release begins supporting running containers with the Singularity Runtime. This may simplify deployments and allow tighter security than Docker.

Remove the "In addition to LSF support" it doesn't add any information.

The second sentence "This may..." is vague and uncertain. Either it should mention/link to a more specific point-by-point comparison of Singularity and Docker or not say anything and assume that the reader knows what they want.

We’ve tested this version against Singularity 3.7.4, and its support should be considered in alpha state. We suggest you check our additional notes about its current limitations.

Like the LSF section this should probably just say "For more information please check our documentation page."

Suggestions applied, thanks.

2.3.0~rc2-1

arvados|57628b973965cdedb28b18523bbe2b98f1ea6e10

arvados-workbench2|d0611f387f0ec0163fc8f6053a7715bc3d592700

build-and-publish-rc-packages: #106

workbench2-build-release-candidate-package: #74

arvados/jobs:2.3.0rc2 docker image created at docker-jobs-image-release: #18

Provision script test successful for 2.3.0~rc2: test-provision: #125

2.3.0~rc3-1

arvados|a598bba05ec26c8103bd7f8cf9e4dcf451469531

build-and-publish-rc-packages: #107

arvados-workbench2|d0611f387f0ec0163fc8f6053a7715bc3d592700

workbench2-build-release-candidate-package: #75

2.3.0~rc4

arvados|067a68b5e9dfa1c7d5e68fd64553e0ced89cad36

build-and-publish-rc-packages: #108

arvados-workbench2|d0611f387f0ec0163fc8f6053a7715bc3d592700

workbench2-build-release-candidate-package: #76

2.3.0

arvados|067a68b5e9dfa1c7d5e68fd64553e0ced89cad36

build-and-publish-rc-packages: #109

arvados-workbench2|d0611f387f0ec0163fc8f6053a7715bc3d592700

workbench2-build-release-candidate-package: #77

arvados-api-server: 2.3.0-1
arvados-client: 2.3.0-1
arvados-controller: 2.3.0-1
arvados-dispatch-cloud: 2.3.0-1
arvados-docker-cleaner: 2.3.0-1
arvados-git-httpd: 2.3.0-1
arvados-health: 2.3.0-1
arvados-server: 2.3.0-1
arvados-src: 2.3.0-1
arvados-sync-groups: 2.3.0-1
arvados-workbench: 2.3.0-1
arvados-workbench2: 2.3.0-1
arvados-ws: 2.3.0-1
crunch-dispatch-local: 2.3.0-1
crunch-dispatch-slurm: 2.3.0-1
crunch-run: 2.3.0-1
crunchstat: 2.3.0-1
keep-balance: 2.3.0-1
keep-block-check: 2.3.0-1
keep-exercise: 2.3.0-1
keepproxy: 2.3.0-1
keep-rsync: 2.3.0-1
keepstore: 2.3.0-1
keep-web: 2.3.0-1
libarvados-perl: 2.3.0-1
libpam-arvados-go: 2.3.0-1
python3-arvados-cwl-runner: 2.3.0-1
python3-arvados-fuse: 2.3.0-1
python3-arvados-python-client: 2.3.0-1
python3-arvados-user-activity: 2.3.0-1
python3-crunchstat-summary: 2.3.0-1
python3-cwltest: 2.3.0-1
python-arvados-cwl-runner: 2.3.0-1
python-arvados-fuse: 2.3.0-1
python-arvados-python-client: 2.3.0-1
python-arvados-user-activity: 2.3.0-1
python-crunchstat-summary: 2.3.0-1
python-cwltest: 2.3.0-1

build-and-publish-rc-packages: #110

The centos7 one needed to re-run: build-publish-packages-testing-centos7: #116

docker-jobs-image-release: #21

build-publish-packages-python-ruby: #41

build-and-release-docker-arvbox-demo-image: #30