Project

General

Profile

Actions

Bug #13517

closed

keepstore memory usage appears to not be related to MaxBuffers

Added by Joshua Randall almost 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Keep
Target version:
Story points:
-
Release:
Release relationship:
Auto

Description

Our keepstore nodes have repeatedly run our instances out of memory when under heavy load, resulting in the OOM killer being invoked.

We started out with MaxBuffers set to be close to the amount of memory on the machine (i.e. 126 * 64MiB = 8064MiB with 8.4GiB on the machine). We then repeatedly reduced it, with apparently no effect on keepstore's memory consumption.

We now have MaxBuffers: 64 (which should be 4GiB of RAM) on an 8.4GiB machine, and it keeps running the instances out of RAM.
Sometimes when the OOM killer decides to kill keepstore, we get a clear record of it being responsible for consuming pretty much all of the RAM on the instance:

[19519.435919] Out of memory: Kill process 13297 (keepstore) score 911 or sacrifice child
[19519.437758] Killed process 13297 (keepstore) total-vm:9871532kB, anon-rss:8014404kB, file-rss:1104kB


Subtasks 1 (0 open1 closed)

Task #13551: Review 13517-buffer-leakResolvedTom Clegg05/31/2018Actions
Actions

Also available in: Atom PDF