Project

General

Profile

Actions

Feature #4042

closed

[Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)

Added by Tom Clegg over 9 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Crunch
Target version:
Story points:
1.0

Description

Example job patterns that should be supported:
  • 6 directories, each with 8 pairs of fastq files → 48 tasks, each operating on one pair
  • 3 dirs with 1 pair of fastq files each + 3 dirs with 4 pairs of fastq files each → 15 tasks, each operating on one pair
  • 6 dirs, each with 8 bam files → 6 tasks, each operating on one dir
  • 6 dirs, each with 1 bam file, plus a static list of chromosomes → 6×23 tasks, each operating on one {bam file, chromosome}
  • 6 dirs, each with 23 bam files → 6×23 tasks, each operating on one file
  • 6 dirs, each with 23 bam files → 6 tasks, each operating on (all of the bam files in) one dir

The output should preserve the input directory structure: in each of the above examples, the output collection should have 6 directories with the same names as the 6 input directories.


Subtasks 1 (0 open1 closed)

Task #4103: Review 4042-run-command-MxNResolvedPeter Amstutz10/06/2014Actions

Related issues

Has duplicate Arvados - Feature #3501: Ability to run one pipeline over a set of inputsClosedActions
Actions

Also available in: Atom PDF