Actions
Idea #14923
open
[CWL] Reduce memory footprint by using webdav to get file/dir listings in collections (instead of Python SDK)
Start date:
Due date:
Story points:
-
Release:
Release relationship:
Auto
Description
The hope is to reduce arvados-cwl-runner's memory requirement enough to run several a-c-r containers at once on a single low-cost node (see #14922).
Depending on how much work this is to implement/test, it may be worth first instrumenting a-c-r and predicting the impact on memory footprint and runtime (e.g., webdav cache hits will be slower than in-process Python SDK cache hits).
Updated by
Actions