Project

General

Profile

Actions

Bug #21223

closed

arv-mount sets RLIMIT_NOFILE sufficient to --file-cache

Added by Peter Amstutz 3 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
FUSE
Story points:
-
Release relationship:
Auto

Description

The Python keep client limits itself to using 1/4 of all file handles. Because each slot requires two file handles, this means the maximum number of cache blocks is RLIMIT_NOFILE / 8.

If RLIMIT_NOFILE is less than 10240, arv-mount will adjust RLIMIT_NOFILE to 10240.

This means, unless RLIMIT_NOFILE is adjusted before invoking arv-mount, the maximum disk cache arv-mount can have in typical usage is 1280 blocks or 80 GiB.

We have a user that needs more than 80 GiB of cache, but setting --file-cache to a larger value silently fails.

arv-mount should choose a value of RLIMIT_NOFILE that is adequate to fulfill --file-cache and fail if it cannot.


Subtasks 1 (0 open1 closed)

Task #21240: Review 21223-arv-mount-nofileResolvedPeter Amstutz11/29/2023Actions
Actions #1

Updated by Peter Amstutz 3 months ago

  • Status changed from New to In Progress
Actions #2

Updated by Peter Amstutz 3 months ago

  • Assigned To set to Peter Amstutz
  • Category set to FUSE
  • Description updated (diff)
  • Subject changed from Increase arv-mount filehandle limit to arv-mount sets RLIMIT_NOFILE sufficient to --file-cache
Actions #3

Updated by Peter Amstutz 3 months ago

  • Release set to 67
Actions #7

Updated by Peter Amstutz 3 months ago

21223-arv-mount-nofile @ 7982e6ae73cc314954a86514bf54b10c38ee592d

Tests linked above

  • All agreed upon points are implemented / addressed.
    • comments
  • Anything not implemented (discovered or discussed during work) has a follow-up story.
    • chooses RLIMIT_NOFILE based on --file-cache
  • Code is tested and passing, both automated and manual, what manual testing was done is described
    • tested manually, fixed a minor logging issue and made it so that RLIMIT_NOFILE is logged on normal operation and confirmed the value was set as expected.
  • Documentation has been updated.
    • n/a
  • Behaves appropriately at the intended scale (describe intended scale).
    • makes it possible to have more than 1280 cache blocks, so that we can cache inputs in the 100s of gigabytes range
  • Considered backwards and forwards compatibility issues between client and server.
    • n/a
  • Follows our coding standards and GUI style guidelines.
    • yes

This is very simple, if an explicit --file-cache is provided, ensure that RLIMIT_NOFILE is set appropriately so that the desired size of --file-cache is actually possible.

Actions #8

Updated by Brett Smith 3 months ago

Peter Amstutz wrote in #note-7:

21223-arv-mount-nofile @ 7982e6ae73cc314954a86514bf54b10c38ee592d

Is it possible to please get a few tests for this? It looks like it should be straightforward to mock resource.setrlimit and check it's called correctly for no/low/high --file-cache.

Should we add more handling the case where the NOFILE limit we want exceeds the hard limit? The current code takes care not to request more than that, but it seems like that situation would be confusing for the user the way exceeding the soft limit is now: they're not actually gonna get the cache size they requested. We could at least warn about it or something.

Casting to int is fine but for the future if you want it, you can get C-style integer division with the // operator.

Thanks.

Actions #10

Updated by Peter Amstutz 3 months ago

Brett Smith wrote in #note-8:

Peter Amstutz wrote in #note-7:

21223-arv-mount-nofile @ 7982e6ae73cc314954a86514bf54b10c38ee592d

Is it possible to please get a few tests for this? It looks like it should be straightforward to mock resource.setrlimit and check it's called correctly for no/low/high --file-cache.

Added test_default_file_cache, test_small_file_cache, test_large_file_cache and test_file_cache_hard_limit.

Should we add more handling the case where the NOFILE limit we want exceeds the hard limit? The current code takes care not to request more than that, but it seems like that situation would be confusing for the user the way exceeding the soft limit is now: they're not actually gonna get the cache size they requested. We could at least warn about it or something.

I added a warning.

Casting to int is fine but for the future if you want it, you can get C-style integer division with the // operator.

Thanks.

21223-arv-mount-nofile @ a05e443dbfcde94651afe783e633b08d79e2b6d1

developer-run-tests: #3950

Actions #11

Updated by Brett Smith 3 months ago

Peter Amstutz wrote in #note-10:

21223-arv-mount-nofile @ a05e443dbfcde94651afe783e633b08d79e2b6d1

This LGTM, thank you.

Actions #12

Updated by Peter Amstutz 3 months ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF