Story #14923
[CWL] Reduce memory footprint by using webdav to get file/dir listings in collections (instead of Python SDK)
Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Start date:
Due date:
% Done:
0%
Estimated time:
Story points:
-
Description
The hope is to reduce arvados-cwl-runner's memory requirement enough to run several a-c-r containers at once on a single low-cost node (see #14922).
Depending on how much work this is to implement/test, it may be worth first instrumenting a-c-r and predicting the impact on memory footprint and runtime (e.g., webdav cache hits will be slower than in-process Python SDK cache hits).
History
#1
Updated by Peter Amstutz 3 months ago
Updated by - Target version deleted (
To Be Groomed)
