Story #8662

[GATK Queue] Support separate runtime constraints for different tools

Added by Brett Smith over 5 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Brett Smith
Category:
-
Target version:
Start date:
03/09/2016
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
0.5

Description

In a GATK Queue Job, runtime_constraints should just be sufficient to run the Queue process itself. It should find runtime constraints for individual tools in the Job's script_parameters like:

"script_parameters": {
  "runtime_constraints": {
    "value": {
      "RealignerTargetCreator": { "min_cores_per_node": 4, … },
      "IndelRealigner":         { "min_cores_per_node": 1, … },
      …
    }
  },
  …
}

Implementation:

  1. In ArvadosJobRunner.scala, capture the tool name explicitly (the tool is already matched from the command line using the regxes on lines 213-217)
  2. Use the tool name as a key into job.script_parameters.runtime_contraints.
  3. Update the top level runtime constraints with any keys supplied the tool-specific runtime constraints.
  4. Use the updated runtime constraints to submit the job using that tool.

Subtasks

Task #8670: Review gatk-protected branch 8662-separate-runtime-constraints-wipResolvedBrett Smith

History

#1 Updated by Brett Smith over 5 years ago

  • Status changed from New to In Progress
  • Assigned To set to Brett Smith
  • Target version changed from 2016-03-30 sprint to 2016-03-16 sprint
  • Story points set to 0.5

#2 Updated by Brett Smith over 5 years ago

  • Description updated (diff)

#3 Updated by Brett Smith over 5 years ago

Our test job wx7k5-8i9sb-i1e9xo681jdzfwm shows the new code working as intended. There was a failure at the lower levels.

#4 Updated by Peter Amstutz over 5 years ago

8662-separate-runtime-constraints-wip @ commit:df47b15 LGTM

#5 Updated by Bryan Cosca over 5 years ago

The test job has completed after fixing the failures before: wx7k5-8i9sb-7dkisf8ad84prqv

#6 Updated by Brett Smith over 5 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF