Feature #16583

Can programmatically distinguish between final outputs (results of top-level containers) and intermediate outputs.

Added by Peter Amstutz 7 months ago. Updated 13 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Crunch
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Output collections have properties "container_request: <uuid>" and "type: output". This allows workbench2 to offer a filter on outputs, logs, and general.

Similar to how I can filter on just top level containers, I'd like to be able to filter on top level outputs. We also want to be able to do scripted cleaning up of intermediate outputs while leaving the final output alone.

Top level containers have "requested_by_container_uuid: null". Technically we could look up the container request for each collection to determine if it is top level or not, but that can't be written as a single query filter.

Proposed approach:

  1. Add an output_properties field on container_request which are properties that will be set on the final output collection (similar to output_name and output_ttl).
  2. Arvados-cwl-runner gains the ability to set properties on collections, has convention for marking intermediate and final collections.
  3. Add a rake task that the admin can optionally use to infer the intermediate and final collection output property from requested_by_container_uuid.
  4. Add to Workbench2 the ability to filter out intermediate collections from project display.

History

#1 Updated by Peter Amstutz 7 months ago

  • Description updated (diff)

#2 Updated by Peter Amstutz 6 months ago

  • Target version changed from 2020-08-12 Sprint to 2020-08-26 Sprint

#3 Updated by Peter Amstutz 6 months ago

  • Target version changed from 2020-08-26 Sprint to 2020-09-09 Sprint

#4 Updated by Peter Amstutz 5 months ago

  • Target version changed from 2020-09-09 Sprint to 2020-09-23 Sprint

#5 Updated by Peter Amstutz 4 months ago

  • Target version changed from 2020-09-23 Sprint to 2020-10-07 Sprint

#6 Updated by Peter Amstutz 4 months ago

  • Target version changed from 2020-10-07 Sprint to 2020-10-21 Sprint

#7 Updated by Peter Amstutz 3 months ago

  • Target version changed from 2020-10-21 Sprint to 2020-11-04 Sprint

#8 Updated by Peter Amstutz 3 months ago

  • Description updated (diff)

#9 Updated by Peter Amstutz 3 months ago

  • Target version changed from 2020-11-04 Sprint to 2020-11-18

#10 Updated by Peter Amstutz 3 months ago

  • Target version changed from 2020-11-18 to 2020-11-04 Sprint

#11 Updated by Peter Amstutz 3 months ago

  • Target version changed from 2020-11-04 Sprint to 2020-11-18

#12 Updated by Peter Amstutz 3 months ago

  • Target version changed from 2020-11-18 to 2020-12-02 Sprint

#13 Updated by Peter Amstutz 2 months ago

  • Target version changed from 2020-12-02 Sprint to 2020-12-16 Sprint

#14 Updated by Peter Amstutz about 2 months ago

  • Target version changed from 2020-12-16 Sprint to 2021-01-06 Sprint

#15 Updated by Peter Amstutz about 1 month ago

  • Target version changed from 2021-01-06 Sprint to 2021-01-20 Sprint

#16 Updated by Peter Amstutz 13 days ago

  • Target version changed from 2021-01-20 Sprint to 2021-02-03 Sprint

Also available in: Atom PDF