Feature #6310

[FUSE] Support scaling the internal block cache based on number of open files

Added by Brett Smith over 3 years ago. Updated over 3 years ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
FUSE
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

FUSE's current caching strategy can serve you well if you're reading files sequentially, or a bunch of small files. If you're interleaving reads across many large files, you can potentially exhaust the cache and thrash it endlessly.

For these sorts of use cases, FUSE would perform better if it kept a small number of blocks in the cache for each file it has open.

The exact implementation should be hashed out. One possibility: the current limit becomes the maximum size of the entire cache. FUSE tracks which blocks belong to which file(s). When it comes time to evict blocks from the cache, FUSE figures out which file(s) have the most blocks cached, and from that list, evicts the least recently used block.


Related issues

Related to Arvados - Story #3640: [SDKs] Add runtime option to SDKs (esp Python and arv-mount) to use a filesystem directory block cache as an alternative to RAM cache.New

Related to Arvados - Feature #8228: [SDKs] [FUSE] Python SDK and arv-mount use Range requests when a caller requests part of a block that has been ejected from the cacheNew2016-01-19

History

#1 Updated by Brett Smith over 3 years ago

  • Description updated (diff)
  • Category set to FUSE

Also available in: Atom PDF