Idea #3706
Updated by Tom Clegg over 9 years ago
Background: Crunch job tasks can use Keep as a cache for manifest fragments, which crunch-job retrieves at the end of the job using @arv-get {block-locator}@. The @arv-get@ program does not know at this point whether the supplied @block-locator@ is a reference to a block to retrieve from Keep or a collection to retrieve from the API server.
Current behavior: arv-get issues a warning.
* <pre>Sat Sep 27 02:00:42 2014 2014-09-27 02:00:41 arvados.collection[7714] WARNING: \
API server did not return Collection '4de3386d281fec68fdc91c31d5ffd3fe+51502+Axxx@54380491'. Trying to fetch directly from Keep (deprecated).
</pre>
Desired behavior:
* If @arv-get@ is asked to retrieve something that looks like a block hash and has a permission signature hint, it should try fetching it from Keep _first_, then issue a warning and fall back on API server.
* If @arv-get@ is asked to retrieve something that looks like a block hash and has _no_ permission signature hint, it should try fetching it from API server, then fall back on Keep. If both fail, the warning should indicate that both methods were attempted and both failed. The existing warning ("plan A failed, trying plan B now") should be removed, i.e., if Keep succeeds, no warning is printed.
** This scenario is useful only when keepstore services are configured to permit anonymous unauthenticated read access. Otherwise, it wastes an HTTP request but is otherwise harmless.