Project

General

Profile

Actions

Feature #20532

open

Grouping steps by sample

Added by Peter Amstutz 10 months ago. Updated 9 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
CWL
Target version:
Story points:
-

Description

When using a workflow that performs a top-level scatter over samples, we want to be able to monitor the progress of each sample independently.

Currently, all the steps are in a flat list, and the step names are de-duplicated simply by appending numbers.

What I think the user needs is a view that is sample oriented, which hides (to the extent possible) implementation details such as whether it is executed as 1 workflow = 1 sample or 1 workflow = 10 samples or 1 workflow = 1000 samples.

Need to think about what the sample oriented view needs to show.

In general, there's a need to make the workflow orchestration lighter weight (each workflow/subworkflow instance takes up fewer resources than an entire compute node) when scattering over a large batch of samples.


Related issues

Related to Arvados - Feature #20825: Extension to run subworkflow in its own runnerResolvedPeter Amstutz10/13/2023Actions
Actions #1

Updated by Peter Amstutz 10 months ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz 10 months ago

  • Target version changed from To be scheduled to Future
Actions #3

Updated by Peter Amstutz 9 months ago

  • Description updated (diff)
Actions #4

Updated by Peter Amstutz 8 months ago

  • Related to Feature #20825: Extension to run subworkflow in its own runner added
Actions

Also available in: Atom PDF