Arvados

By design, Data Manager fixed underreplicated blocks by sending "pull lists" to the keepstore servers.

Currently, Data Manager computes lists of underreplicated blocks, but doesn't send the pull lists.

• data manager must send its pull lists to keep servers
• data manager must have at least one comprehensive integration test covering a miniature version of normal operation:
  • bring up api, keepstore, and keepproxy services
  • store some collections
  • delete some blocks (using keepstore DELETE method with a suitable token) so the desired replication level is no longer met for some collections
  • run data manager
  • wait for all keepstores to finish working their trash and pull lists (by getting each keepstore's /status.json until the queues are empty and workers are idle)
  • make sure all collections' blocks are back up to their desired replication level
  • run data manager again
  • wait for all keepstores to finish working their trash and pull lists
  • make sure no more (excess) replicas have been made