Project

General

Profile

Actions

Idea #9528

closed

[Crunch v2] SLURM dispatcher should not use `sbatch --parsable`

Added by Brett Smith over 8 years ago. Updated over 8 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Crunch
Target version:
Start date:
06/30/2016
Due date:
Story points:
0.5

Description

This switch was added in SLURM 14.03.0rc1, which is relatively new. Of all our supported distros, only Debian jessie includes a new-enough version. That's probably too new.


Subtasks 2 (0 open2 closed)

Task #9564: Review 9528-slurm-parsableResolvedRadhika Chippada06/30/2016Actions
Task #9566: Drop --parsable flagResolvedTom Clegg06/30/2016Actions
Actions #1

Updated by Brett Smith over 8 years ago

(03:59:06 PM) tomclegg: I think we could regexp stdout for "Submitted batch job \d+" to accomplish the same thing.
(04:01:22 PM) Me: https://github.com/SchedMD/slurm/blob/master/NEWS → --parsable was added in Slurm 14.03.0rc1
(04:02:58 PM) Me: Only jessie has that, none of our other supported Debian or Ubuntu versions do.
(04:03:03 PM) Me: (And CentOS doesn't package SLURM at all.)
(04:04:30 PM) tomclegg: regexp is sounding pretty good, then. Prefer "a bit clunky" to "doesn't work on most platforms"...
(04:07:16 PM) Me: Right. If our goal is to drop into existing SLURM clusters, we'll drop into more if we avoid newer features.
(04:08:44 PM) Me: So before we start playing whack-a-mole with these sorts of bugs, maybe we should figure out what version of SLURM we want to target?
(04:09:24 PM) Me: Ubuntu 12.04 has 2.3, and it's going to be supported for another year.
(04:10:13 PM) Me: That's also what wheezy has.
(04:10:41 PM) Me: (Though admittedly wheezy is only supported unofficially now, but it seemed worth double-checking.)
(04:14:41 PM) Me: CentOS 5 came out in 2007, it's still supported for another year. SLURM's NEWS file doesn't go back far enough to tell me what version corresponds to that time.
(04:15:26 PM) Me: CentOS 6 corresponds with SLURM 2.2.
(04:16:01 PM) Me: So... 2.3 seems reasonable, 2.2 would be extra-safe.

Actions #2

Updated by Tom Clegg over 8 years ago

This regexp should match any reasonable variant of the known human-readable ("not --parsable") output format.

/(?i)(job|id) *(\d+)/
Actions #3

Updated by Brett Smith over 8 years ago

  • Target version set to 2016-07-20 sprint
Actions #4

Updated by Brett Smith over 8 years ago

  • Story points set to 0.5
Actions #5

Updated by Tom Clegg over 8 years ago

  • Category set to Crunch
  • Assigned To set to Tom Clegg
Actions #6

Updated by Tom Clegg over 8 years ago

Turns out we don't even need a regexp. crunch-dispatch-slurm reads sbatch's stdout but then just throws it away.

9528-slurm-parsable @ cba9e59 drops the --parsable option, and logs sbatch's stdout ("sbatch succeeded: %s") in case it's useful for debugging.

Actions #7

Updated by Tom Clegg over 8 years ago

  • Status changed from New to In Progress
Actions #8

Updated by Radhika Chippada over 8 years ago

LGTM

Actions #9

Updated by Tom Clegg over 8 years ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados|commit:edbd43f2b89a915cf33b963c1e1ae86447c1e93b.

Actions

Also available in: Atom PDF