Project

General

Profile

Actions

Idea #19923

closed

keep-balance experimental single-chunk mode

Added by Tom Clegg about 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Story points:
2.0
Release relationship:
Auto

Description

Background: These features will enable us to collect data about the impact of sharding on keep-balance time and memory requirements. Our expectation is that sharding will ultimately enable keep-balance to scale by a few more orders of magnitude, and decrease its resource consumption for all cluster sizes.

Features:
  • If a hexadecimal prefix is specified on the command line (e.g., "f" or "0f"), keep-balance only balances blocks with that prefix.
    • This only makes sense as a data-gathering experiment using the -once flag.
    • Statistics will be misleading, because they will still have labels like "total" but only report a small percentage of the blocks.
    • At this stage we will not support an option to automatically run all shards sequentially or concurrently.
  • Improve progress reporting while reading indexes from keepstore servers: e.g., every 5 minutes log "{volume}: received {N} blocks so far"
  • Periodically report keep-balance's own memory usage

Subtasks 1 (0 open1 closed)

Task #19943: Review 19923-keep-balance-chunkResolvedTom Clegg01/27/2023Actions
Actions

Also available in: Atom PDF