Project

General

Profile

Actions

Idea #12289

closed

Migrate to Slurm API

Added by Tom Morris over 6 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Low
Assigned To:
-
Category:
Git hosting
Target version:
-
Story points:
-

Description

Currently nodemanager & crunch-dispatch-slurm parse the output of Slurm command line utilities to get the information that it needs. This is fragile and labor intensive since changes to the output format must be tracked by the parsing code.

We should investigate switching to using the actual Slurm API.
https://github.com/PySlurm/pyslurm

Actions #1

Updated by Tom Morris over 6 years ago

  • Description updated (diff)
Actions #2

Updated by Tom Clegg over 6 years ago

Potential benefits include better compatibility across slurm versions (less CLI-output-parsing code) and lower overhead (less shelling out to CLI).

However, https://github.com/PySlurm/pyslurm says

You will need to instruct the setup.py script where either the Slurm install root directory or where the Slurm libraries and Slurm include files are

It sounds like this package provides an API client for a specific version of slurm, which could be just another distribution/deployment nightmare. It only makes sense if every (current+future) slurm library version is forward & backward compatible with all slurm versions. Otherwise, CLI tools are more reliable -- at least it's possible to accommodate versions with "if v3 then X else Y" when necessary.

Actions #3

Updated by Peter Amstutz over 2 years ago

  • Priority changed from Normal to Low
  • Status changed from New to Closed
  • Category set to Git hosting
Actions #4

Updated by Ward Vandewege over 2 years ago

  • Target version deleted (Arvados Future Sprints)
Actions

Also available in: Atom PDF