Arvados

A container in "queued" state with priority 0 can technically be restarted and is currently displayed in Workbench 2 as "on hold".

It can technically be started by raising the priority to non-zero, so there is a "Run" button.

This is extremely confusing. All priority 0 container requests should be shown as either "cancelling" (if the container hasn't ended yet, e.g. Locked or Running) or "cancelled" (when nothing is running).

This behavior has also caused problems for the user.

They submitted a workflow, and it created a workflow step that was committed but never started.

They cancelled the workflow, which (under the current logic) put the container "on hold".

Later, they submitted mostly the same workflow, but with a change to the keep cache setting.

Currently, we don't consider keep cache as part of container reuse.

As a result, the first "on hold" container was reused, but with a different keep_cache setting than the user asked for the 2nd time.

As a result, the user got different a different keep_cache setting than the expected.

We should probably revisit the role of keep cache in container reuse decisions, but if the first container had been cancelled instead of put on hold, it would have also avoided this problem.