Story #16578

Scripted setup of the tutorial cluster by copying data from data cluster

Added by Peter Amstutz over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Sample Pipelines
Target version:
Start date:
09/09/2020
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Subtasks

Task #16765: Review 16578-copy-tutorialResolvedPeter Amstutz


Related issues

Related to Arvados - Bug #17960: [arv-copy] document ability to copy projectsResolved09/08/2021

Associated revisions

Revision 3f18ac30
Added by Peter Amstutz about 1 year ago

Merge branch '16578-copy-tutorial' refs #16578

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <>

History

#1 Updated by Peter Amstutz over 1 year ago

  • Assigned To set to Peter Amstutz

#2 Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2020-07-15 to 2020-08-12 Sprint

#3 Updated by Peter Amstutz over 1 year ago

  • Assigned To deleted (Peter Amstutz)

#4 Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2020-08-12 Sprint to 2020-08-26 Sprint

#5 Updated by Peter Amstutz about 1 year ago

  • Target version changed from 2020-08-26 Sprint to 2020-09-09 Sprint

#6 Updated by Peter Amstutz about 1 year ago

  • Assigned To set to Peter Amstutz

#7 Updated by Peter Amstutz about 1 year ago

  • Category set to Sample Pipelines

#8 Updated by Peter Amstutz about 1 year ago

  • Status changed from New to In Progress

#9 Updated by Peter Amstutz about 1 year ago

16578-copy-tutorial @ b07b70d9024e7203f4cae3bbc526fc36e9cca19e

Adds project copying feature to arv-copy. This copies collections and workflows contained in the source project.

Adds tools/copy-tutorial/copy-tutorial.sh

This is a simple script that uses arv-copy project copying to copy tutorials from the data cluster (jutro).

Also partially addresses a long-standing efficiency bug in copying large collections, building up the new manifest that will be sent to the destination cluster no longer involves geometric growth in string copying.

https://ci.arvados.org/view/Developer/job/developer-run-tests/2074/

#10 Updated by Peter Amstutz about 1 year ago

  • Target version changed from 2020-09-09 Sprint to 2020-09-23 Sprint

#11 Updated by Nico César about 1 year ago

  • Target version deleted (2020-09-23 Sprint)

I red the code at b07b70d9024e7203f4cae3bbc526fc36e9cca19e and seems fine but tests are missing for the new paramenters

I noticed sdk/python/tests/test_arv_copy.py is kind of skinny and probably to test this would be a combination of arv-put, arv-copy then arv-get

Is there any way we can test this easily? Just to have more coverage in case we change something down the road (or some 3rd party dependency does!)

#12 Updated by Nico César about 1 year ago

  • Target version set to 2020-09-23 Sprint

#13 Updated by Peter Amstutz about 1 year ago

Nico César wrote:

I red the code at b07b70d9024e7203f4cae3bbc526fc36e9cca19e and seems fine but tests are missing for the new paramenters

I noticed sdk/python/tests/test_arv_copy.py is kind of skinny and probably to test this would be a combination of arv-put, arv-copy then arv-get

Is there any way we can test this easily? Just to have more coverage in case we change something down the road (or some 3rd party dependency does!)

16578-copy-tutorial @ 3489a9b5ce083cd17dae6d50c8c5ceeee29b074b

Added a copy project (which includes copy collection) integration test.

https://ci.arvados.org/view/Developer/job/developer-run-tests/2081/

#14 Updated by Nico César about 1 year ago

LGTM, ready to merge

#15 Updated by Peter Amstutz about 1 year ago

  • Status changed from In Progress to Resolved

#16 Updated by Peter Amstutz about 1 year ago

  • Release set to 25

#17 Updated by Ward Vandewege 2 months ago

  • Related to Bug #17960: [arv-copy] document ability to copy projects added

Also available in: Atom PDF