Arvados

In the case of potential hardware corruption, we would like to be able to thoroughly check that a collection is intact.

the keep-block-check tool currently takes a list of blocks and uses HEAD requests to see if the blocks exist.

extend this with a mode to do deep validation of a collection:

1. get the manifest text and check for valid syntax/grammar
2. compute the portable data hash and compare against the collection record
3. go through each block and fetch from keep, compute the hash of the block against the expected hash. report any blocks that are missing or return an error.