Idea #12289
closedMigrate to Slurm API
Description
Currently nodemanager & crunch-dispatch-slurm parse the output of Slurm command line utilities to get the information that it needs. This is fragile and labor intensive since changes to the output format must be tracked by the parsing code.
We should investigate switching to using the actual Slurm API.
https://github.com/PySlurm/pyslurm
Updated by Tom Clegg over 6 years ago
Potential benefits include better compatibility across slurm versions (less CLI-output-parsing code) and lower overhead (less shelling out to CLI).
However, https://github.com/PySlurm/pyslurm says
You will need to instruct the setup.py script where either the Slurm install root directory or where the Slurm libraries and Slurm include files are
It sounds like this package provides an API client for a specific version of slurm, which could be just another distribution/deployment nightmare. It only makes sense if every (current+future) slurm library version is forward & backward compatible with all slurm versions. Otherwise, CLI tools are more reliable -- at least it's possible to accommodate versions with "if v3 then X else Y" when necessary.
Updated by Peter Amstutz over 2 years ago
- Priority changed from Normal to Low
- Status changed from New to Closed
- Category set to Git hosting
Updated by Ward Vandewege over 2 years ago
- Target version deleted (
Arvados Future Sprints)