Idea #6309
Updated by Brett Smith almost 9 years ago
For jobs that access many files simultaneously, FUSE's default block cache is usually not sufficient, and will thrash regularly. This can lead to blocks downloaded multiple times, and degraded performance.
If there's evidence that it will substantially benefit production pipelines, add a runtime_constraint to jobs so they can specify an argument for the @--file-cache@ option of their FUSE mount.
If the performance benefit isn't big enough, we don't want to do this, for all the usual reasons we don't want to avoid more development on our current JSON pipeline templates and Crunch.
Bryan will run some benchmarks and report on how helpful this can be.