Project

General

Profile

Actions

Feature #6310

open

[FUSE] Support scaling the internal block cache based on number of open files

Added by Brett Smith over 9 years ago. Updated 10 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
FUSE
Target version:
Story points:
-
Release:
Release relationship:
Auto

Description

FUSE's current caching strategy can serve you well if you're reading files sequentially, or a bunch of small files. If you're interleaving reads across many large files, you can potentially exhaust the cache and thrash it endlessly.

For these sorts of use cases, FUSE would perform better if it kept a small number of blocks in the cache for each file it has open.

The exact implementation should be hashed out. One possibility: the current limit becomes the maximum size of the entire cache. FUSE tracks which blocks belong to which file(s). When it comes time to evict blocks from the cache, FUSE figures out which file(s) have the most blocks cached, and from that list, evicts the least recently used block.


Related issues 2 (1 open1 closed)

Related to Arvados - Idea #3640: [SDKs] Add runtime option to SDKs (esp Python and arv-mount) to use a filesystem directory block cache as an alternative to RAM cache.ClosedActions
Related to Arvados - Feature #8228: [SDKs] [FUSE] Python SDK and arv-mount use Range requests when a caller requests part of a block that has been ejected from the cacheNewActions
Actions #1

Updated by Brett Smith over 9 years ago

  • Description updated (diff)
  • Category set to FUSE
Actions #2

Updated by Ward Vandewege over 3 years ago

  • Target version deleted (Arvados Future Sprints)
Actions #3

Updated by Peter Amstutz almost 2 years ago

  • Release set to 60
Actions #4

Updated by Peter Amstutz 10 months ago

  • Target version set to Future
Actions

Also available in: Atom PDF