Feature #8017
closed
[Crunch2] support runtime constraints
Description
Propagate runtime constraints from Container to slurm in crunch-dispatch-slurm.
See runtime_constraints at Containers API
We should support (at least) the following cases:- request 12 GB RAM, do not specify number of CPU cores: {"ram":12000000000}
- request 12 GB RAM and 4 CPU cores: {"ram":12000000000, "vcpus":4}
- do not propagate any constraints: {}
The Container record's runtime_constraints hash is expected to have a numeric value (or null/missing) for each element -- never a range of acceptable values like the ContainerRequest examples.
Every Container record is guaranteed to have a value for "ram" and "vcpus" by the time it's eligible for dispatch.
Updated by Tom Clegg almost 8 years ago
- Assigned To set to Peter Amstutz
- Story points set to 1.0
Updated by Radhika Chippada almost 8 years ago
- Assigned To changed from Peter Amstutz to Radhika Chippada
Updated by Peter Amstutz almost 8 years ago
Updated by Radhika Chippada almost 8 years ago
- Status changed from New to In Progress
Updated by Radhika Chippada almost 8 years ago
The story description said: (1) The Container record's runtime_constraints hash is expected to have a numeric value for each element, (2) Every Container record is guaranteed to have a value for "ram" and "vcpus" by the time it's eligible for dispatch.
- Based on this, I updated the slurm dispatch code to expect these numeric runtime_constraints in the Container record and pass them to sbatch command.
- I used strconv.Itoa to pass the numeric values as strings to sbatch. Please let me know if this needs any corrections.
Commit f1adedeb
Updated by Peter Amstutz almost 8 years ago
A careful reading of the SLURM page suggests that we should be using --mem-per-cpu instead of --mem. So, you need to divide ram by the number of cpus and round up. Also, the slurm --mem-per-cpu option takes MegaBytes but the crunch v2 API gives RAM in bytes. You need to divide by 2^20 and round up.
Updated by Radhika Chippada almost 8 years ago
Thanks Peter. Updated per note 9 at b5e2bbb9
Updated by Peter Amstutz almost 8 years ago
One more tweak, this should be int64 instead of int because int in Go can be either 32 or 64 bits.
RuntimeConstraints map[string]int `json:"runtime_constraints"`
Updated by Radhika Chippada almost 8 years ago
Updated to RuntimeConstraints to use int64. Thanks.
Updated by Radhika Chippada almost 8 years ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|commit:a32c69b81296860a30cc33909226d9294f411adf.