Feature #11410

[SDK] tool for copying CWL file + input collections

Added by Peter Amstutz 10 months ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
0.5

Description

arv-copy can copy workflow records (+ dependencies). However, our current workflow is oriented around CWL files stored in a git repo + input documents with keep references. There is a gap in the tooling to make it easy to copy all the dependencies (reference collections, docker images, input collections) from one cluster to another starting from a git repo of CWL files and input documents.

Suggest an mode for arvados-cwl-runner which specifies one or more "source" Arvados sites. During workflow initialization, if a collection or docker image is not available on the site on which the workflow will run, copy it from one of the "source sites". Arvados-cwl-runner already has similar behavior to upload files and Docker images from the local file system.

History

#1 Updated by Peter Amstutz 10 months ago

  • Description updated (diff)

#2 Updated by Peter Amstutz 10 months ago

  • Description updated (diff)

#3 Updated by Tom Morris 5 months ago

  • Target version set to Arvados Future Sprints

Also available in: Atom PDF