Feature #19847
closed
Choose disk cache size based on RAM request
Added by Peter Amstutz over 1 year ago.
Updated over 1 year ago.
Release relationship:
Auto
Description
Instead of applying a flat 8 GB disk cache size it should be proportional to the task size.
After giving it some thought, let's try
2 GiB <= the size of the RAM request <= 32 GiB
- Status changed from New to In Progress
- Description updated (diff)
- Subject changed from Choose disk cache size based on ram/cpu count to Choose disk cache size based on RAM request
19847-cwl-disk-cache-size @ 0531cc9f35e78a7be1eec7eb96fb6cc668ebcefa
- Default disk cache heuristic computed by arv-mount is now
2 GiB <= the size of the RAM request <= 32 GiB
- Added arvados-cwl-runner CWL extension to explicitly select between ram and disk cache
- Fixed crunch-run to pass
--ram-cache to explicitly select RAM cache when specified by runtime_constraints
developer-run-tests: #3402 
- Target version changed from 2022-12-07 Sprint to 2022-12-21 Sprint
This seems to mean the DefaultKeepCacheDisk config is essentially a boolean (all non-zero values just mean "enable disk cache, choose size automatically") when it comes to containers submitted by arvados-cwl-runner -- is that right? Would this be worth mentioning in the config comment?
Tom Clegg wrote in #note-6:
This seems to mean the DefaultKeepCacheDisk config is essentially a boolean (all non-zero values just mean "enable disk cache, choose size automatically") when it comes to containers submitted by arvados-cwl-runner -- is that right? Would this be worth mentioning in the config comment?
If I add that to the config comment would this be a LGTM?
It does seem like a weird knob. I suppose that's OK if it's documented. But I wonder if it's worth trying to make it less weird.
- rename config to
MinimumKeepCacheDisk, instead of using hard-coded 2 GiB in a-c-r?
- config sets the default disk cache size as % RAM size instead of # bytes?
- auto-size logic moves to railsapi so it applies to top-level a-c-r containers too?
- Status changed from In Progress to Resolved
Also available in: Atom
PDF
Updated by 
Updated by