Project

General

Profile

Idea #13429

Updated by Tom Clegg almost 6 years ago

arvados-cwl-runner should accept a command line option that determines the storage classes used to save the workflow output. 

 The minimal implementation is to update the storage_classes_desired field of the output collection. It's OK that However, this wastes storage resources by first writing the output data to the default class, then copying it to the desired class, and then (after blobSignatureTTL) deleting it from default. 

 To avoid this: 
 * Populate a (new) "output_storage_classes" field when creating container requests. 
 * Propagate that field to container where possible. This isn't always possible: one container can satisfy two CRs with different desired storage classes. In such cases, we can pick one arbitrarily or use "default". 
 * In crunch-run, propagate desired classes through arv-mount, and through the Go SDK when creating an output collection, as needed. 

Back