Feature #8017

[Crunch2] support runtime constraints

Added by Peter Amstutz over 5 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Radhika Chippada
Category:
-
Target version:
Start date:
05/03/2016
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
1.0
Release:
Release relationship:
Auto

Description

Propagate runtime constraints from Container to slurm in crunch-dispatch-slurm.

See runtime_constraints at Containers API

We should support (at least) the following cases:
  • request 12 GB RAM, do not specify number of CPU cores: {"ram":12000000000}
  • request 12 GB RAM and 4 CPU cores: {"ram":12000000000, "vcpus":4}
  • do not propagate any constraints: {}

The Container record's runtime_constraints hash is expected to have a numeric value (or null/missing) for each element -- never a range of acceptable values like the ContainerRequest examples.

Every Container record is guaranteed to have a value for "ram" and "vcpus" by the time it's eligible for dispatch.


Subtasks

Task #9088: Implement in slurm dispatcherResolvedRadhika Chippada

Task #9087: Specify minimum constraintsResolvedPeter Amstutz

Task #9089: Review branch: 8017-slurm-runtime-constraintsResolvedPeter Amstutz

Associated revisions

Revision a32c69b8
Added by Radhika Chippada almost 5 years ago

closes #8017
Merge branch '8017-slurm-runtime-constraints'

History

#1 Updated by Tom Clegg about 5 years ago

  • Target version set to 2016-05-11 sprint

#2 Updated by Tom Clegg about 5 years ago

  • Assigned To set to Peter Amstutz
  • Story points set to 1.0

#3 Updated by Radhika Chippada about 5 years ago

  • Assigned To changed from Peter Amstutz to Radhika Chippada

#4 Updated by Tom Clegg about 5 years ago

  • Description updated (diff)

#5 Updated by Peter Amstutz about 5 years ago

http://slurm.schedmd.com/sbatch.html

--mem=<MB>

--cpus-per-task=<ncpus>

#6 Updated by Tom Clegg about 5 years ago

  • Description updated (diff)

#7 Updated by Radhika Chippada about 5 years ago

  • Status changed from New to In Progress

#8 Updated by Radhika Chippada about 5 years ago

The story description said: (1) The Container record's runtime_constraints hash is expected to have a numeric value for each element, (2) Every Container record is guaranteed to have a value for "ram" and "vcpus" by the time it's eligible for dispatch.

  • Based on this, I updated the slurm dispatch code to expect these numeric runtime_constraints in the Container record and pass them to sbatch command.
  • I used strconv.Itoa to pass the numeric values as strings to sbatch. Please let me know if this needs any corrections.

Commit f1adedeb

#9 Updated by Peter Amstutz almost 5 years ago

A careful reading of the SLURM page suggests that we should be using --mem-per-cpu instead of --mem. So, you need to divide ram by the number of cpus and round up. Also, the slurm --mem-per-cpu option takes MegaBytes but the crunch v2 API gives RAM in bytes. You need to divide by 2^20 and round up.

#10 Updated by Radhika Chippada almost 5 years ago

Thanks Peter. Updated per note 9 at b5e2bbb9

#11 Updated by Peter Amstutz almost 5 years ago

One more tweak, this should be int64 instead of int because int in Go can be either 32 or 64 bits.

    RuntimeConstraints map[string]int `json:"runtime_constraints"`

#12 Updated by Radhika Chippada almost 5 years ago

Updated to RuntimeConstraints to use int64. Thanks.

#13 Updated by Radhika Chippada almost 5 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:a32c69b81296860a30cc33909226d9294f411adf.

Also available in: Atom PDF