Project

General

Profile

Actions

Story #19132

open

Versioning registered workflows

Added by Peter Amstutz 9 months ago. Updated 24 days ago.

Status:
In Progress
Priority:
Normal
Assigned To:
-
Target version:
-
Start date:
03/01/2023
Due date:
09/30/2023 (Due in about 8 months)
% Done:

0%

Estimated time:
Story points:
-
Release:
Release relationship:
Auto

Description

Support versioning for registered workflows in Workbench.

Possible features:

  • When launching a workflow, user can choose among multiple versions
  • Can easily determine which version of the workflow was used to run a particular pipeline.
  • Text of workflow that is used for execution is more like the text that the user started from (instead of workflow packing).
  • Text of workflow as submitted (i.e. original) is also saved so that users can compare the "original" they submitted against their VCS later (good for auditability).
  • Requirement from customer: when making a copy, record the upstream collection it was copied from, indicate in the user interface that the upstream version has changed, offer to bump up to current version.

Possible design:

  • Store CWL in a collection with "type: workflow" property
  • Use collection versioning
  • Store workflow metadata in properties
    • entry point
    • input spec
    • git commit/branch/repo? if uploaded a git directory
    • docker jobs image to use
  • Deprecate "workflow" records
  • Identify the workflow version used for a given pipeline by matching the PDH

Implementation:

  • arvados-cwl-runner support for creating/updating new-style registered workflows
  • Workbench 2 support (this is probably the majority of the work)
    • picking
    • viewing
    • launching

Proposed new behavior for a-c-r: always create a workflow collection when running a workflow, so we have accessible record of what was run.


Related issues

Related to Arvados - Story #19464: record git information when registering a workflowResolvedPeter Amstutz09/08/2022

Actions
Actions

Also available in: Atom PDF