Bug #17759

[Keep] azure blob driver 404 errors are weird

Added by Ward Vandewege 12 months ago. Updated 11 months ago.

Assigned To:
Target version:
Start date:
Due date:
% Done:


Estimated time:
Story points:


The Azure blob driver errors when Keep requests a blob that does not exist are a bit wonky. Here's an example:

Jun 04 17:45:50 keep29.su92l.arvadosapi.com keepstore[29504]: time="2021-06-04T17:45:50Z" level=warning msg="error in Compare(faf52080a91d4e914a08c2531926e0d6) on volume azure-storage-container:\"su92l-keep-0\"" error="storage: service returned error: StatusCode=404, ErrorCode=BlobNotFound, ErrorMessage=The specified blob does not exist.\nRequestId:9147e360-e01e-0063-5e69-59cc7e000000\nTime:2021-06-04T17:45:50.3375250Z, RequestInitiated=Fri, 04 Jun 2021 17:45:49 GMT, RequestId=9147e360-e01e-0063-5e69-59cc7e000000, API Version=2018-03-28, QueryParameterName=, QueryParameterValue=" 

These lines appear just before every PUT request (is this the client doing a GET first? Or the Keep Azure driver doing a GET before a PUT?).

Compare with the AWS version, which does not seem to appear before every PUT:

Jun 04 02:45:48 keep0.2xpu4.arvadosapi.com keepstore[17414]: {"PID":17414,"RequestID":"req-1g7wpq8lwq1od109q80m","level":"info","msg":"request","remoteAddr":"","reqBytes":0,"reqForwardedFor":"","reqHost":"keep0.2xpu4.arvadosapi.com:25107","reqMethod":"GET","reqPath":"REDACTEDPDH+67006287+REDACTEDSIGNATURE","reqQuery":"","time":"2021-06-04T02:45:48.239614800Z"}
Jun 04 02:46:03 keep0.2xpu4.arvadosapi.com keepstore[17414]: {"PID":17414,"RequestID":"req-1g7wpq8lwq1od109q80m","level":"info","msg":"response","remoteAddr":"","reqBytes":0,"reqForwardedFor":"","reqHost":"keep0.2xpu4.arvadosapi.com:25107","reqMethod":"GET","reqPath":"REDACTEDPDH+67006287+REDACTEDSIGNATURE","reqQuery":"","respBody":"Not Found\n","respBytes":10,"respStatus":"Not Found","respStatusCode":404,"time":"2021-06-04T02:46:03.963565926Z","timeToStatus":15.723936,"timeTotal":15.723947,"timeWriteBody":0.000010}

Can we make the Azure blob driver error for a 404 more straightforward?


#1 Updated by Ward Vandewege 12 months ago

  • Description updated (diff)

#2 Updated by Ward Vandewege 12 months ago

  • Description updated (diff)

#3 Updated by Ward Vandewege 12 months ago

  • Description updated (diff)

#4 Updated by Peter Amstutz 11 months ago

  • Target version deleted (To Be Groomed)

Also available in: Atom PDF