Project

General

Profile

Actions

Feature #6611

closed

[Data Manager] Act on pull lists for underreplicated blocks

Added by Tom Clegg almost 7 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
-
Category:
Keep
Target version:
-
Start date:
07/14/2015
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Story points:
2.0

Description

By design, Data Manager fixed underreplicated blocks by sending "pull lists" to the keepstore servers.

Currently, Data Manager computes lists of underreplicated blocks, but doesn't send the pull lists.

Before this can be deployed:
  • data manager must send its pull lists to keep servers
  • data manager must have at least one comprehensive integration test covering a miniature version of normal operation:
    • bring up api, keepstore, and keepproxy services
    • store some collections
    • delete some blocks (using keepstore DELETE method with a suitable token) so the desired replication level is no longer met for some collections
    • run data manager
    • wait for all keepstores to finish working their trash and pull lists (by getting each keepstore's /status.json until the queues are empty and workers are idle)
    • make sure all collections' blocks are back up to their desired replication level
    • run data manager again
    • wait for all keepstores to finish working their trash and pull lists
    • make sure no more (excess) replicas have been made

Subtasks 2 (2 open0 closed)

Task #6612: Report trash and pull queue & worker status in keepstore /status.jsonNew07/14/2015

Actions
Task #6631: Review 6611-send-pull-listIn Progress07/15/2015

Actions

Related issues

Related to Arvados - Story #9162: [Block Manager] Trashes unreferenced blocks (with mtime > TTL if desired) and overreplicated blockResolvedTom Clegg05/16/2016

Actions
Actions #1

Updated by Tom Clegg almost 7 years ago

  • Description updated (diff)
Actions #2

Updated by Tom Clegg almost 7 years ago

  • Description updated (diff)
Actions #3

Updated by Tom Clegg almost 7 years ago

  • Description updated (diff)
Actions #4

Updated by Peter Amstutz almost 7 years ago

Note: there is already a preliminary 6611-send-pull-list branch that I wrote when I wrote at the same time as the "send trash list" code.

Actions #5

Updated by Peter Amstutz about 6 years ago

  • Target version changed from Arvados Future Sprints to 2016-05-11 sprint
Actions #6

Updated by Tom Clegg about 6 years ago

Fixing underreplicated blocks is a special case of rebalancing: "rebalance only the blocks that have less than desired replication."

If
  • block B has replication A ("actual"), and
  • block B should have replication D ("desired"), and
  • A < D, and
  • keep service K is one of the first D services in the probe order for block A, and
  • block B is not in service K's index
...then
  • pick some service K2 that does have block B in its index
  • add K2/B to the pull list for K.
Actions #7

Updated by Radhika Chippada about 6 years ago

  • Assigned To set to Radhika Chippada
Actions #8

Updated by Radhika Chippada about 6 years ago

  • Story points set to 2.0
Actions #9

Updated by Brett Smith about 6 years ago

  • Target version changed from 2016-05-11 sprint to Arvados Future Sprints
Actions #10

Updated by Radhika Chippada about 6 years ago

  • Assigned To deleted (Radhika Chippada)
Actions #11

Updated by Tom Clegg over 5 years ago

  • Status changed from New to Closed
Actions #12

Updated by Tom Morris over 5 years ago

  • Target version deleted (Arvados Future Sprints)
Actions

Also available in: Atom PDF