Idea #21377
Updated by Peter Amstutz 4 months ago
We are seeing a pattern where a command is wrapped by a simple workflow in order to provide a CWL interface. In the current architecture, this means launching a container with the CWL runner, then having it submit another container request to run the actual command. This is slow (it has to wait for another node to up come) and wasteful (the workflow runner node is completely idle while this is happening). It also requires that the user click through to a second level of container request to see the log of the actual command.
I propose a CWL extension to specify that a CommandLineTool should be launched in the same container as the workflow runner.
This will start the command, wait for it, and collect the outputs. This can probably borrow heavily from the CWL job runner.
This feature will have the following restrictions:
* It must use the same container image (e.g. the image must contain the tool, a-c-r, and nodejs if there are any Javascript expressions)
* It shares hardware resources (mostly RAM) with a-c-r
It should be able to infer resource usage using similar same logic as RunInSingleContainer.