Project

General

Profile

Actions
Copy link

Bug #17390

closed

Set storage classes for intermediates and final outputs

Added by Nico César about 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Peter Amstutz
Category:
CWL
Target version:
2021-08-18 sprint
Story points:
-
Release:
Arvados 2.3.0
Release relationship:
Auto

Description

  --storage-classes STORAGE_CLASSES
                        Specify comma separated list of storage classes to be
                        used when saving workflow output to Keep.

base casecase

arvados-cwl-runner --name "CWL diagnostics hasher" --create-workflow hasher-workflow.cwl hasher-workflow-job.yml --storage-classes STORAGE_CLASSES

Existing behavior is final output collection goes to the storage class.

Currently all intermediates go to 'default'.

Expected behaviour:

  • Need to be able to set the storage class of output of a container request
  • Should be able to choose a storage class and have it used for intermediate results through the workflow
  • Need to be able to specify the storage class for a given workflow step ???

In CWL:

StorageClassHint:
  storageClasses: [foo, bar]

Turns this into "output_storage_classes" on the container request.

Command line parameter applies to everything.

Want to support a case where all the intermediates go to fast storage and only the final outputs go into long term storage.

Subtasks 1 (0 open — 1 closed)

Task #17976: Review 17390-cwl-storage-classesResolvedPeter Amstutz08/12/2021Actions

Related issues

Related to Arvados Epics - Idea #16107: Storage classesResolved03/01/202109/30/2021Actions
Blocked by Arvados - Feature #17349: [Keep API] Revisit "replicas_desired" and "storage_classes_desired".RejectedActions
Actions
Copy link

Also available in: Atom PDF