Story #6309

[Crunch] Support a runtime_constraint that specifies how much RAM arv-mount should use for cache

Added by Brett Smith about 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Crunch
Target version:
Start date:
11/24/2015
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
1.0

Description

For jobs that access many files simultaneously, FUSE's default block cache is usually not sufficient, and will thrash regularly. This can lead to blocks downloaded multiple times, and degraded performance.

If there's evidence that it will substantially benefit production pipelines, add a runtime_constraint to jobs so they can specify an argument for the --file-cache option of their FUSE mount.

Implementation

  • When the job specifies a keep_cache_mb_per_task runtime constraint, crunch-job calls arv-mount with that value in the --file-cache switch (converting units as needed).
  • Document this new runtime constraint in the Jobs schema API reference.

Subtasks

Task #7791: Review 6309-file-cache-runtime-constraintResolvedPeter Amstutz

Task #7790: Write docsResolvedPeter Amstutz

Task #7862: Update crunch-jobResolvedPeter Amstutz


Related issues

Related to Arvados - Bug #7747: [FUSE] arv-mount is too slow for strelkaNew11/10/2015

Associated revisions

Revision 8788c145
Added by Peter Amstutz over 3 years ago

Merge branch '6309-file-cache-runtime-constraint' closes #6309

History

#1 Updated by Brett Smith about 4 years ago

  • Subject changed from [Maybe] [Crunch] Support a runtime_constraint that specifies how much RAM arv-mount should use for cache to [Maybe] [Crunch] Support a runtime_constraint that specifies how much RAM arv-mount should use for cache
  • Description updated (diff)
  • Category set to Crunch

#2 Updated by Brett Smith about 4 years ago

  • Description updated (diff)

#3 Updated by Brett Smith about 4 years ago

  • Target version changed from Arvados Future Sprints to Deferred

Bryan's benchmarks indicate that this change doesn't help performance, at least for the cases we're worried about. Deferring this until there's real demand or we have CWL instead, whichever comes first.

#4 Updated by Peter Amstutz almost 4 years ago

  • Target version deleted (Deferred)

#5 Updated by Brett Smith almost 4 years ago

  • Target version set to 2015-12-02 sprint

#6 Updated by Brett Smith almost 4 years ago

  • Subject changed from [Maybe] [Crunch] Support a runtime_constraint that specifies how much RAM arv-mount should use for cache to [Crunch] Support a runtime_constraint that specifies how much RAM arv-mount should use for cache
  • Description updated (diff)

#7 Updated by Brett Smith almost 4 years ago

  • Assigned To set to Peter Amstutz

#8 Updated by Peter Amstutz almost 4 years ago

6309-file-cache-runtime-constraint ready for review

#9 Updated by Peter Amstutz almost 4 years ago

  • Status changed from New to In Progress

#10 Updated by Brett Smith over 3 years ago

Reviewing 292b28f.

  • Could $arv_file_cache just be the empty string when the constraint is not specified? That way, we'll just get arv-mount's default, without duplicating it.
  • The documentation update duplicates the row about min_ram_mb_task. Please clean that up.

Thanks.

#11 Updated by Peter Amstutz over 3 years ago

Brett Smith wrote:

Reviewing 292b28f.

  • Could $arv_file_cache just be the empty string when the constraint is not specified? That way, we'll just get arv-mount's default, without duplicating it.

Done.

  • The documentation update duplicates the row about min_ram_mb_task. Please clean that up.

Fixed.

Now at dcab751

#12 Updated by Brett Smith over 3 years ago

Peter Amstutz wrote:

Now at dcab751

This is good to merge. Thanks.

#13 Updated by Peter Amstutz over 3 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 67 to 100

Applied in changeset arvados|commit:8788c145b860e19a1f04c4dc6abdcda14c859403.

Also available in: Atom PDF