Idea #9653
Updated by Tom Clegg over 6 years ago
It's hard to predict the right keepstore's default max-buffers setting for @-max-buffers@. * The sysadmin must coordinate configuration files with host characteristics: e.g., when upgrading is 128. This means it's willing to a bigger node, the sysadmin must also remember allocate up to change max-buffers at 8GB to hold blocks that it's shuttling around. If the same time. * The sysadmin must account for (and/or override) Go's garbage collection threshold, likely 100% (default) or 10% (our recommendation). * The sysadmin must predict how much box doesn't have 8GB available, keepstore can run out of memory keepstore will use for purposes other than {buffers + 10% garbage collection overhead}, and reduce crash. Improve the default max-buffers accordingly. setting to avoid crashing. Possible approaches (needs discussion): * The sysadmin must predict how much memory will be used by other processes running on the same node (e.g., cloud/monitoring/orchestration agents). In addition to Use a static limit on very low setting, like 8 (.5GB). Out of the number of buffers, we could configure a maximum keepstore memory size, and check actual process memory usage before allocating a buffer. This isn't perfect (it doesn't limit allocations box, the performance would only be suitable for anything other than buffers) test servers, but it would be strictly safer than the current behavior, which _never_ accounts for other allocations. Memory use could be limited by providing one or more of these configs: at least avoid crashing. <pre> # Maximum memory use, as * Select a number of MiB MaxMemoryMiB: 6000 # Maximum memory use, as a default based on some percentage of the box's total system RAM RAM. 50%? 80%? 25%? MaxMemoryPercent: 80 </pre> Suggested default: <pre> MaxMemoryPercent: 80 </pre> To address the last difficulty (predicting memory use by other processes) we could also offer * Don't have a MinMemoryFree or MinMemoryAvailable config, which avoids allocating more buffers when Linux reports a low MemFree or MemAvailable. More than one limit can be configured at once. Keepstore should allocate new buffers only when none of the configured limits will be exceeded. default, and instead refuse to run if this setting isn't provided.