Story #12289
Migrate to Slurm API
0%
Description
Currently nodemanager & crunch-dispatch-slurm parse the output of Slurm command line utilities to get the information that it needs. This is fragile and labor intensive since changes to the output format must be tracked by the parsing code.
We should investigate switching to using the actual Slurm API.
https://github.com/PySlurm/pyslurm
Subtasks
History
#1
Updated by Tom Morris over 3 years ago
- Description updated (diff)
#2
Updated by Tom Clegg over 3 years ago
Potential benefits include better compatibility across slurm versions (less CLI-output-parsing code) and lower overhead (less shelling out to CLI).
However, https://github.com/PySlurm/pyslurm says
You will need to instruct the setup.py script where either the Slurm install root directory or where the Slurm libraries and Slurm include files are
It sounds like this package provides an API client for a specific version of slurm, which could be just another distribution/deployment nightmare. It only makes sense if every (current+future) slurm library version is forward & backward compatible with all slurm versions. Otherwise, CLI tools are more reliable -- at least it's possible to accommodate versions with "if v3 then X else Y" when necessary.