Idea #9528
closed[Crunch v2] SLURM dispatcher should not use `sbatch --parsable`
Description
This switch was added in SLURM 14.03.0rc1, which is relatively new. Of all our supported distros, only Debian jessie includes a new-enough version. That's probably too new.
Updated by Brett Smith over 8 years ago
(03:59:06 PM) tomclegg: I think we could regexp stdout for "Submitted batch job \d+" to accomplish the same thing.
(04:01:22 PM) Me: https://github.com/SchedMD/slurm/blob/master/NEWS → --parsable was added in Slurm 14.03.0rc1
(04:02:58 PM) Me: Only jessie has that, none of our other supported Debian or Ubuntu versions do.
(04:03:03 PM) Me: (And CentOS doesn't package SLURM at all.)
(04:04:30 PM) tomclegg: regexp is sounding pretty good, then. Prefer "a bit clunky" to "doesn't work on most platforms"...
(04:07:16 PM) Me: Right. If our goal is to drop into existing SLURM clusters, we'll drop into more if we avoid newer features.
(04:08:44 PM) Me: So before we start playing whack-a-mole with these sorts of bugs, maybe we should figure out what version of SLURM we want to target?
(04:09:24 PM) Me: Ubuntu 12.04 has 2.3, and it's going to be supported for another year.
(04:10:13 PM) Me: That's also what wheezy has.
(04:10:41 PM) Me: (Though admittedly wheezy is only supported unofficially now, but it seemed worth double-checking.)
(04:14:41 PM) Me: CentOS 5 came out in 2007, it's still supported for another year. SLURM's NEWS file doesn't go back far enough to tell me what version corresponds to that time.
(04:15:26 PM) Me: CentOS 6 corresponds with SLURM 2.2.
(04:16:01 PM) Me: So... 2.3 seems reasonable, 2.2 would be extra-safe.
Updated by Tom Clegg over 8 years ago
This regexp should match any reasonable variant of the known human-readable ("not --parsable") output format.
/(?i)(job|id) *(\d+)/
Updated by Tom Clegg over 8 years ago
- Category set to Crunch
- Assigned To set to Tom Clegg
Updated by Tom Clegg over 8 years ago
Turns out we don't even need a regexp. crunch-dispatch-slurm reads sbatch's stdout but then just throws it away.
9528-slurm-parsable @ cba9e59 drops the --parsable option, and logs sbatch's stdout ("sbatch succeeded: %s") in case it's useful for debugging.
Updated by Tom Clegg over 8 years ago
- Status changed from In Progress to Resolved
Applied in changeset arvados|commit:edbd43f2b89a915cf33b963c1e1ae86447c1e93b.