Project

General

Profile

Idea #19132

Updated by Peter Amstutz over 2 years ago

Support versioning for registered workflows in Workbench. 

 Possible features: 

 * When launching a workflow, user can choose among multiple versions 
 * Can easily determine which version of the workflow was used to run a particular pipeline. 
 * Text of workflow that is used for execution is more like the text that the user started from (instead of workflow packing). 
 * Text of workflow as submitted (i.e. original) is also saved so that users can compare the "original" they submitted against their VCS later (good for auditability). 
 * Requirement from customer: when making a copy, record the upstream collection it was copied from, indicate in the user interface that the upstream version has changed, offer to bump up to current version. 

 Possible design: 

 * Store CWL in a collection with "type: workflow" property  
 * Use collection versioning 
 * Store workflow metadata in properties 
 ** entry point 
 ** input spec 
 ** git commit/branch/repo?    if uploaded a git directory 
 ** docker jobs image to use 
 * Deprecate Are "workflow" records 
 even needed any more?    Could be kept in a collection with "type: workflow" property  
 * Identify the workflow version used for a given pipeline by matching the PDH  

 Implementation: 

 * arvados-cwl-runner support for creating/updating new-style registered workflows 
 * Workbench 2 support (this is probably the majority of the work) 
 ** picking 
 ** viewing 
 ** launching 

Back