Project

General

Profile

Actions

Story #19132

open

Versioning registered workflows

Added by Peter Amstutz 7 months ago. Updated 9 days ago.

Status:
In Progress
Priority:
Normal
Assigned To:
-
Target version:
-
Start date:
09/01/2022
Due date:
03/31/2023 (Due in about 4 months)
% Done:

0%

Estimated time:
Story points:
-
Release:
Release relationship:
Auto

Description

Support versioning for registered workflows in Workbench.

Possible features:

  • When launching a workflow, user can choose among multiple versions
  • Can easily determine which version of the workflow was used to run a particular pipeline.
  • Text of workflow that is used for execution is more like the text that the user started from (instead of workflow packing).
  • Text of workflow as submitted (i.e. original) is also saved so that users can compare the "original" they submitted against their VCS later (good for auditability).
  • Requirement from customer: when making a copy, record the upstream collection it was copied from, indicate in the user interface that the upstream version has changed, offer to bump up to current version.

Possible design:

  • Store CWL in a collection with "type: workflow" property
  • Use collection versioning
  • Store workflow metadata in properties
    • entry point
    • input spec
    • git commit/branch/repo? if uploaded a git directory
    • docker jobs image to use
  • Deprecate "workflow" records
  • Identify the workflow version used for a given pipeline by matching the PDH

Implementation:

  • arvados-cwl-runner support for creating/updating new-style registered workflows
  • Workbench 2 support (this is probably the majority of the work)
    • picking
    • viewing
    • launching

Proposed new behavior for a-c-r: always create a workflow collection when running a workflow, so we have accessible record of what was run.

Actions #1

Updated by Peter Amstutz 7 months ago

  • Start date set to 05/15/2022
  • Due date set to 08/15/2022
Actions #2

Updated by Peter Amstutz 7 months ago

  • Description updated (diff)
Actions #3

Updated by Peter Amstutz 7 months ago

  • Start date changed from 05/15/2022 to 05/23/2022
Actions #4

Updated by Ward Vandewege 7 months ago

  • Description updated (diff)
Actions #5

Updated by Peter Amstutz 7 months ago

  • Description updated (diff)
Actions #6

Updated by Peter Amstutz 7 months ago

  • Release set to 28
Actions #7

Updated by Peter Amstutz 7 months ago

  • Description updated (diff)
Actions #8

Updated by Peter Amstutz 6 months ago

  • Due date changed from 08/15/2022 to 09/14/2022
Actions #9

Updated by Peter Amstutz 6 months ago

  • Start date changed from 05/23/2022 to 07/01/2022
  • Due date changed from 09/14/2022 to 09/30/2022
Actions #10

Updated by Peter Amstutz 4 months ago

  • Start date changed from 07/01/2022 to 08/31/2022
  • Due date changed from 09/30/2022 to 10/31/2022
Actions #11

Updated by Peter Amstutz 4 months ago

  • Start date changed from 08/31/2022 to 09/01/2022
Actions #12

Updated by Peter Amstutz about 2 months ago

  • Due date changed from 10/31/2022 to 12/31/2022
Actions #13

Updated by Peter Amstutz about 1 month ago

  • Status changed from New to In Progress
Actions #14

Updated by Peter Amstutz 9 days ago

  • Due date changed from 12/31/2022 to 03/31/2023
Actions

Also available in: Atom PDF