Feature #8551
Updated by Peter Amstutz about 8 years ago
Node manager currently shuts down nodes based on three criteria: is the node idle, is the queue empty, and is the shutdown window open.
Unfortunately, this doesn't take into account recent activity. It is a common development pattern to run a job, make some tweaks, and then re-submit it. It is also common for pipeline runners to have gap in time between submitting jobs as it looks at the output of a previous job and builds the next job.
Node manager should have a configurable option to set a minimum time the node must be idle before shutting it down, to address these kinds of race conditions where a node gets (inconveniently) shut down the middle of actual work.