Bug #6157
Updated by Tom Clegg almost 9 years ago
h3. background
Changing slurm config files, and keeping them synchronized across controller+workers, is a bit painful and can cause race conditions that are annoying to diagnose, so we try to avoid setups where it has to change during normal operation.
"computeN", where N is decimal, lets you write compute[0-123] in your slurm config files. Therefore, nodes.ping makes it easy to manage a setup like this.
* nodes.ping automatically assigns a slot number to each node record when it sends its first ping, and sets hostname=computeN at the same time (see bug #6156). The node is expected to look at the ping response and change its hostname to the assigned name.
* if your api's max_compute_nodes config matches your slurm config, nodes.ping will not assign hostnames that aren't defined in your slurm config.
*However,* in some setups it might be inconvenient/difficult/impossible to use computeN.* as hostnames.
h3. improvement
Install docs should include a section explaining
* Why compute[0-N] is a good idea (see above)
* What to do differently if you use a different naming scheme (e.g., your worker nodes' hostnames are {alice, bob, clay, ...})
We should make the simplifying assumption that the hostnames are assigned manually/OOB, and known in advance. IOW, instead of covering scenarios where slurm config has to change every time a new compute node is turned up, we should just advise against that.
AFAIK(TC), as long as the available/powered-on nodes' hostnames are a subset of the hostnames given in slurm.conf, and no two hosts have the same name, slurm and Arvados should work without any code changes. Even #6156 has a workaround: when you add a node record, set its slot_number at the same time you assign its hostname.