Bug #14766
open
ResourceRequirement disk space ask should be shown in workbench / API response
Description
"runtime_constraints": {
"vcpus": 16,
"ram": 26214400000
},
This should also include the amount of scratch asked for.
For example: https://workbench.e51c5.arvadosapi.com/container_requests/e51c5-xvhdp-3wyuwgbbej342l1
On workbench it shows
runtime_constraints: keep_cache_ram 268435456 ram 26214400000 vcpus 16
Updated by Bryan Cosca over 5 years ago
- Subject changed from tmpdirMin ResourceRequirement should be shown in workbench / API response to ResourceRequirement disk space ask should be shown in workbench / API response
Updated by Tom Clegg about 5 years ago
The scratch size used to choose a suitable node type is computed by the dispatcher (it's a function of "tmp" mount sizes and approximate docker image size) which isn't quite the same thing as the CWL ResourceRequirement number.
It looks like Workbench could get the ResourceRequirement number by looking at the size specified for the /var/spool/cwl tmp mount. In the given example, the API response tab shows this was 135524253696 (~126 GiB).
Currently the minimum scratch space number used to choose a VM type isn't saved or logged anywhere (unless slurm is in charge of choosing node types, in which case it's mentioned in the sbatch ... --tmp=%d ... command line in system logs). We could log it to system logs in the cloud scenario too easily enough -- but in both cases logging it to the container log where a user could see it will require a bit more work because currently the dispatcher doesn't have a way to write to the container log.
Would it be enough to show the CWL ResourceRequirement, or do we also need to log/show the actual minimum scratch space used to choose a node type?
Updated by Bryan Cosca about 5 years ago
Tom Clegg wrote:
Would it be enough to show the CWL ResourceRequirement, or do we also need to log/show the actual minimum scratch space used to choose a node type?
I think both would be useful in this case. I would like to know what I asked for to make sure it matches what my CWL says, and also what the difference was for what I asked for and what was actually computed to choose a node type. That way its very obvious if I'm hitting some edge case where if I lower the scratch space ResourceRequirement by n bytes, I can be on a lower cost node type.
Updated by