Arvados

Feature #14322

Updated by Peter Amstutz over 5 years ago

Arvados-cwl-runner should allow allows users to provide uuids in input documents.    For example: 

 <pre> 
 { 
   "file1": { 
     "class": "File", 
     "location": "keep:zzzzz-4zz18-zzzzzzzzzzzzz/file1.txt" 
   } 
 } 
 </pre> 

 Arvados-cwl-runner should replace the value in "location" with the portable data hash and On execution, record the UUID in the "arv:collection" field.    This applies to both File and Directory objects.    Implementation note: this should probably happen somewhere collection name in @upload_workflow_deps()@ which is responsible for updating File references for uploaded files. arvados extension fields. 

 <pre> 
 { 
   "file1": { 
     "class": "File", 
     "location": "keep:abc+123/file1.txt", 
     "arv:collection": "zzzzz-4zz18-zzzzzzzzzzzzz/file1.txt" 
   } 
 } 
 </pre> 

 Users may provide both "location" by PDH and "arv:collection" in the input.    If both portable data hash and arv:collection are present, the portable data hash will take precedence.    Print a warning (or error???) if the collection uuid is not readable or the Ensure that recording these values does not match the PDH that was provided. 

 When constructing break container requests, if "arv:collection" is known for a File or Directory object, include it in the mount object alongside the portable data hash. 

 Implementation note: code paths that test @startswith("keep:")@ will need to be updated to distinguish keep identifiers by UUID or PDH.    (Alternately, we could use a different uri scheme for uuids). 

 Stretch goals (to propagate uuids through): 

 Uploaded files have "arv:collection" set. 

 When collecting outputs from a container request, set "arv:collection" on the File and Directory objects sourced from the container request output. reuse.

Back