Feature #14322

[CWL] Accept collection uuid in input

Added by Peter Amstutz 2 months ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
2.0

Description

Arvados-cwl-runner should allow users to provide uuids in input documents. For example:

{
  "file1": {
    "class": "File",
    "location": "keep:zzzzz-4zz18-zzzzzzzzzzzzz/file1.txt" 
  }
}

Arvados-cwl-runner should replace the value in "location" with the portable data hash and record the UUID in the "arv:collection" field. This applies to both File and Directory objects. Implementation note: this should probably happen somewhere in upload_workflow_deps() which is responsible for updating File references for uploaded files.

{
  "file1": {
    "class": "File",
    "location": "keep:abc+123/file1.txt",
    "arv:collection": "zzzzz-4zz18-zzzzzzzzzzzzz/file1.txt" 
  }
}

Users may provide both "location" by PDH and "arv:collection" in the input. If both portable data hash and arv:collection are present, the portable data hash will take precedence. Print a warning (or error???) if the collection uuid is not readable or the does not match the PDH that was provided.

When constructing container requests, if "arv:collection" is known for a File or Directory object, include it in the mount object alongside the portable data hash.

Implementation note: code paths that test startswith("keep:") will need to be updated to distinguish keep identifiers by UUID or PDH. (Alternately, we could use a different uri scheme for uuids).


Related issues

Related to Arvados - Feature #11442: [CWL] Resolve symbolic names to collectionsNew

Related to Arvados - Feature #14323: [API] Accept container mounts that specify both uuid and portable_data_hashResolved2018-11-07

History

#1 Updated by Peter Amstutz 2 months ago

  • Status changed from New to In Progress

#2 Updated by Peter Amstutz 2 months ago

  • Subject changed from [CWL] Accept collection uuid to [CWL] Accept collection uuid
  • Status changed from In Progress to New

#3 Updated by Peter Amstutz 2 months ago

  • Description updated (diff)

#4 Updated by Peter Amstutz 2 months ago

  • Subject changed from [CWL] Accept collection uuid to [CWL] Accept collection uuid in input

#5 Updated by Tom Clegg 2 months ago

  • Related to Feature #11442: [CWL] Resolve symbolic names to collections added

#6 Updated by Tom Clegg 2 months ago

  • Related to Feature #14323: [API] Accept container mounts that specify both uuid and portable_data_hash added

#7 Updated by Peter Amstutz 2 months ago

  • Description updated (diff)

#8 Updated by Peter Amstutz 2 months ago

  • Description updated (diff)

#9 Updated by Tom Morris 2 months ago

  • Target version changed from To Be Groomed to Arvados Future Sprints
  • Story points set to 2.0

Also available in: Atom PDF