Project

General

Profile

Actions

Bug #8702

closed

[Keep] HEAD request should do a checksum of the blob without sending it

Added by Nico César about 8 years ago. Updated almost 8 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
Keep
Target version:
-
Story points:
-

Description

Migrations to different storage volumes often include a verification phase.

we should have a method (preferably HEAD ) that will make a check of the block without actually retrying it.


Related issues

Is duplicate of Arvados - Feature #4122: [Keep] Keepstore respects HTTP HEAD method semanticsResolvedActions
Blocks Arvados - Idea #8724: [Keep] Block validation scriptResolvedRadhika Chippada03/16/2016Actions
Actions #1

Updated by Brett Smith about 8 years ago

  • Project changed from 40 to Arvados
  • Description updated (diff)
Actions #2

Updated by Tom Clegg about 8 years ago

  • Subject changed from [KEEPSTORE] HEAD request should do a checksum of the blob without sending it to [Keep] HEAD request should do a checksum of the blob without sending it
  • Category set to Keep
  • Status changed from New to Feedback

This seems to be done already. Go's HTTP library automatically suppresses the body for HEAD requests, and Keepstore accepts HEAD /{hash}.

tomclegg@shell:~$ time curl --head -H "Authorization: OAuth2 $ARVADOS_API_TOKEN" "http://keep6.c97qk.arvadosapi.com:25107/a59f44608b780c968af43da75cc9d6e8+67108864+Ad9785dc3247c083f49c7524690e140659bfef7b3@56fdbbaa" 
HTTP/1.1 200 OK
Content-Length: 67108864
Content-Type: application/octet-stream
Date: Fri, 18 Mar 2016 00:11:29 GMT

real    0m2.036s
user    0m0.000s
sys     0m0.014s

Note: curl -X HEAD is buggy (as of curl 7.35.0): it forgets that method=HEAD affects response handling. When it sees Content-Length: (non-zero) in the response header it just sits around waiting for bytes to arrive, which is correct for non-HEAD requests but incorrect for HEAD.

Actions #3

Updated by Nico César about 8 years ago

Tom Clegg wrote:

This seems to be done already. Go's HTTP library automatically suppresses the body for HEAD requests, and Keepstore accepts HEAD /{hash}.

Is this also doing the checksum check? if so, we can close the ticket and unblock #8724

Actions #4

Updated by Brett Smith almost 8 years ago

  • Status changed from Feedback to Resolved
Actions #5

Updated by Brett Smith almost 8 years ago

HEAD and GET requests use the same handling code, which does a checksum to determine the response code, so yes.

Actions

Also available in: Atom PDF