Feature #18992
Updated by Peter Amstutz about 2 years ago
Currently crunch-run only brings up a local keepstore process if it receives the cluster config on stdin, which currently only happens under arvados-dispatch-cloud. When using slurm or lsf, it is not enabled, and there is no error/warning saying why.
Proposed improvements:
* If cluster config is not supplied on stdin, try reading it from @/etc/arvados/config.yml@ (or a different value specified on the command line via CrunchRunArgumentsList config)
* If local keepstore is enabled in config (LocalKeepBlobBuffersPerVCPU>0) but can't be brought up because cluster config file does not exist or cannot be read, log a message to that effect, and proceed using the usual keepstores
* always log where it got the config from (stdin, somewhere on the file system, or didn't find it, so won't try to use it).
* Explain in config.default.yml comments (and in upgrade notes) that the sysadmin is responsible for deploying the cluster config file to the compute nodes in order to use this feature with slurm or lsf