Bug #16790

[keep-web] S3 ListObjects response should not have empty NextMarker field

Added by Tom Clegg about 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
09/01/2020
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

Currently keep-web returns IsTruncated=false and an empty NextMarker when there is no "next page" to retrieve. NextMarker is also empty when the request has no delimiter.

(According to AWS docs on NextMarker "When response is truncated (the IsTruncated element value in the response is true), you can use the key name in this field as marker in the subsequent request")

However, testing indicates the Amazon .net SDK (github.com/aws/aws-sdk-net) stays in its "next page" loop as long as a NextMarker field is present in the response, even if the NextMarker value is empty and IsTruncated is false.

To address this, and to conform better with the documented behavior when the request has no delimiter ("Note: This element is returned only if you have delimiter request parameter specified."), keep-web should not return the NextMarker tag at all when the value is empty.


Subtasks

Task #16792: Review 16790-next-marker-omitemptyResolvedTom Clegg


Related issues

Related to Arvados Epics - Story #16360: Keep-web supports S3 compatible interfaceResolved07/01/202004/30/2021

Associated revisions

Revision bcee68ee
Added by Tom Clegg about 1 year ago

Merge branch '16790-next-marker-omitempty'

fixes #16790

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <>

History

#1 Updated by Tom Clegg about 1 year ago

  • Project changed from Arvados Private to Arvados

#4 Updated by Lucas Di Pentima about 1 year ago

LGTM, thanks!

#5 Updated by Anonymous about 1 year ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved

#6 Updated by Tom Clegg about 1 year ago

  • Related to Story #16360: Keep-web supports S3 compatible interface added

#7 Updated by Peter Amstutz about 1 year ago

  • Release set to 25

Also available in: Atom PDF