Feature #18163

[Crunch] add CWL extension to limit the number of concurrently running jobs during a scatter

Added by Ward Vandewege about 1 month ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

A typical use case is a CWL workflow that downloads files from a remote server, and scatters over a list of URLs to do so. The remote site does not have infinite bandwidth; the workflow author should be able to specify e.g. downloading no more than 10 URLs concurrently so as not to overwhelm the other side, while scattering over a list of thousands URLs.

History

#1 Updated by Ward Vandewege about 1 month ago

  • Description updated (diff)

Also available in: Atom PDF