Project

General

Profile

Actions

Idea #8662

closed

[GATK Queue] Support separate runtime constraints for different tools

Added by Brett Smith about 8 years ago. Updated about 8 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Story points:
0.5

Description

In a GATK Queue Job, runtime_constraints should just be sufficient to run the Queue process itself. It should find runtime constraints for individual tools in the Job's script_parameters like:

"script_parameters": {
  "runtime_constraints": {
    "value": {
      "RealignerTargetCreator": { "min_cores_per_node": 4, … },
      "IndelRealigner":         { "min_cores_per_node": 1, … },
      …
    }
  },
  …
}

Implementation:

  1. In ArvadosJobRunner.scala, capture the tool name explicitly (the tool is already matched from the command line using the regxes on lines 213-217)
  2. Use the tool name as a key into job.script_parameters.runtime_contraints.
  3. Update the top level runtime constraints with any keys supplied the tool-specific runtime constraints.
  4. Use the updated runtime constraints to submit the job using that tool.

Subtasks 1 (0 open1 closed)

Task #8670: Review gatk-protected branch 8662-separate-runtime-constraints-wipResolvedBrett Smith03/09/2016Actions
Actions #1

Updated by Brett Smith about 8 years ago

  • Status changed from New to In Progress
  • Assigned To set to Brett Smith
  • Target version changed from 2016-03-30 sprint to 2016-03-16 sprint
  • Story points set to 0.5
Actions #2

Updated by Brett Smith about 8 years ago

  • Description updated (diff)
Actions #3

Updated by Brett Smith about 8 years ago

Our test job wx7k5-8i9sb-i1e9xo681jdzfwm shows the new code working as intended. There was a failure at the lower levels.

Actions #4

Updated by Peter Amstutz about 8 years ago

8662-separate-runtime-constraints-wip @ commit:df47b15 LGTM

Actions #5

Updated by Bryan Cosca about 8 years ago

The test job has completed after fixing the failures before: wx7k5-8i9sb-7dkisf8ad84prqv

Actions #6

Updated by Brett Smith about 8 years ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF