Bug #14451

crunchstat-summary recommends 563 TB Keep cache (or more!)

Added by Tom Morris about 1 year ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
03/15/2019
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

Even given the low cache utilization, that seems like a lot

### Summary for make_examples_4 (e51c5-dz642-id66uhhem9viqk0)
# Keep cache miss rate 0.19%
# Keep cache utilization 52.15%
#!! make_examples_4 max RSS was 336 MiB -- try runtime_constraints "ram":1020054732
#!! make_examples_4 Keep cache utilization was 52.15% -- try runtime_constraints "keep_cache_ram":562949953421312 (or more)


Subtasks

Task #14961: Review 14451-crunchstat-summary-cache-ramResolvedTom Morris

Associated revisions

Revision 22bbbebe (diff)
Added by Tom Morris 8 months ago

Fix Keep cache recommendation calculation.

Remove extra MB (2**20) factor. Refs #14451

Arvados-DCO-1.1-Signed-off-by: Tom Morris <>

Revision b3ea1ab1 (diff)
Added by Tom Morris 8 months ago

Fix Keep cache recommendation calculation.

Remove extra MB (2**20) factor. Refs #14451

Arvados-DCO-1.1-Signed-off-by: Tom Morris <>

Revision 7f93b99e (diff)
Added by Tom Morris 8 months ago

Fix Keep cache recommendation calculation.

Remove extra MB (2**20) factor. Refs #14451

Arvados-DCO-1.1-Signed-off-by: Tom Morris <>

Revision d23f9c8a
Added by Tom Morris 8 months ago

Merge branch '14451-crunchstat-summary-cache-ram'

Fix three small crunchstat-summary issues:
- Use default of 1 if no cores requested. fixes #12026
- Don't recommend RAM or CPU changes based on zero usage. fixes #10570
- Remove extra MB (2**20) factor in Keep cache recommendation. Fixes #14451

Arvados-DCO-1.1-Signed-off-by: Tom Morris <>

History

#1 Updated by Tom Clegg 10 months ago

source:tools/crunchstat-summary/crunchstat_summary/summarizer.py probably needs to multiply by self._runtime_constraint_mem_unit()/1048576 when computing asked_mib here:

    def _recommend_keep_cache(self):
...
        asked_mib = self.existing_constraints.get(constraint_key, 256)
...
                asked_mib*2*(2**20)/self._runtime_constraint_mem_unit())

#2 Updated by Tom Morris 8 months ago

  • Assigned To set to Tom Morris
  • Target version set to 2019-03-27 Sprint

#3 Updated by Tom Morris 8 months ago

  • Status changed from New to In Progress

#4 Updated by Tom Morris 8 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

#5 Updated by Tom Morris 6 months ago

  • Release set to 15

Also available in: Atom PDF