Bug #13681
closed
[CWL] Cannot set submit-runner-ram when running from Workbench
Added by Peter Amstutz over 6 years ago.
Updated over 6 years ago.
Release relationship:
Auto
Description
Workbench has the RAM allocation for arvados-cwl-runner jobs hardcoded at 256MB. This is way too stingy (when you run from the command line, the default is 1GB) and users with large collection inputs may need as much as 8GB. When running from workbench, there is no way to set --submit-runner-ram. This should be settable, at minimum when using a-c-r --create-workflow.
Proposed fix:
- Add a "workflow_runner_ram" column to the Workflows table
- When using "a-c-r --create-workflow" or "a-c-r --update-workflow", looks for "--submit-runner-ram" and set "workflow_runner_ram" in the request
- When workbench constructs a container request to run a-c-r, use the value of "workflow_runner_ram" for "--submit-runner-ram" on the a-c-r command line.
Related issues
1 (1 open — 0 closed)
- Status changed from New to In Progress
- Description updated (diff)
- Description updated (diff)
- Status changed from In Progress to New
- Release deleted (
13)
- Release changed from 14 to 13
Is there an appropriate place to put this information in the workflow definition itself, rather than a separate column?
Tom Clegg wrote:
Is there an appropriate place to put this information in the workflow definition itself, rather than a separate column?
Yes, I suppose we could add a new hint for it. Workbench would still need to know how to extract it in order to construct the container request.
- Target version changed from To Be Groomed to 2018-07-18 Sprint
- Story points set to 1.0
- Assigned To set to Peter Amstutz
Reviewing 334c069f4da11278844f4276b5c254e8cbf06412
- There’re some
sdk/cwl test failures on jenkins (already talked about this, I'm just recording the observation)
- Just some code style observations on file
sdk/cwl/arvados_cwl/arvworkflow.py
- Line 55: Could that be replace with this?:
hints = main.get(‘hints’, [])
- The
found var on line 57 could be replaced with an else clause on the for loop?
Lucas Di Pentima wrote:
Reviewing 334c069f4da11278844f4276b5c254e8cbf06412
- There’re some
sdk/cwl test failures on jenkins (already talked about this, I'm just recording the observation)
Yea, I'm going to revert the change which is causing those failures, I filed #13766 to dig deeper.
- Just some code style observations on file
sdk/cwl/arvados_cwl/arvworkflow.py
- Line 55: Could that be replace with this?:
hints = main.get(‘hints’, [])
Yes, I don't know why I wrote it that way.
- The
found var on line 57 could be replaced with an else clause on the for loop?
No, because (according to my research) the "else" clause doesn't fire for empty arrays.
- Related to Bug #13766: [API] bug in libyaml / Pysch used by API server to parse yaml added
Providing that tests pass OK, LGTM.
Regarding the else clause, it seems that they work even with empty lists:
Python 2.7.13 (default, Jun 27 2017, 10:00:24)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> for a in []:
... print "loop"
... else:
... print "else"
...
else
- Status changed from New to Resolved
- % Done changed from 75 to 100
Also available in: Atom
PDF
Updated by 
Updated by Tom Morris 
Updated by 
Updated by