Feature #19088
closed
Expose properties as S3 tags in keep-web
Added by Peter Amstutz almost 2 years ago.
Updated over 1 year ago.
Release relationship:
Auto
Description
Even though S3 doesn't support searching/filtering on tags, applications can still use tags to do client-side filtering.
- Directory placeholder for projects includes properties
- Directory placeholder for collection includes properties
- Every file inside a collection will return that collection's properties
- Despite AWS having a limit of 10 tags, 128 characters for keys and 256 characters for values, we will send all the tags without limits.
- Return the raw ids, no vocabulary interpretation
- Read only
- JSON values (such as lists) are returned in their JSON text representation
- Description updated (diff)
- Target version set to 2022-06-08 sprint
- Target version changed from 2022-06-08 sprint to 2022-05-25 sprint
- Description updated (diff)
- Target version changed from 2022-05-25 sprint to 2022-06-08 sprint
- Target version changed from 2022-06-08 sprint to 2022-06-22 Sprint
- Assigned To set to Tom Clegg
- Target version changed from 2022-06-22 Sprint to 2022-07-06
- Status changed from New to In Progress
Tom Clegg wrote in #note-12:
19088-s3-properties-tags @ dd8f1b0527995bc5ad47710d3a483fa18b827bc6 -- developer-run-tests: #3202 
Is there any particular reason you didn't implement GetBucketTagging and GetObjectTagging?
https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTagging.html
The difference between these and HeadObject seems to be that they returns an XML document rather than embedding it in the headers. Which sounds annoying, but someone might want it, I don't know.
I did some looking, I didn't realize that there's no listing API that can return objects with tags to even be able to do client side filtering efficiently. Clients will have to get the tags of every object separately, which is going to be pretty terrible. Hopefully the keep-web response in that case (once cached) be will be very fast and efficient.
Oops, not really. I just happened to know about the metadata headers and assumed that's what was being asked.
Being readonly, I think it should be just as easy to export the same info in other ways too, if it turns out people really want to use the tagging API.
Tom Clegg wrote in #note-14:
Oops, not really. I just happened to know about the metadata headers and assumed that's what was being asked.
Being readonly, I think it should be just as easy to export the same info in other ways too, if it turns out people really want to use the tagging API.
Per our conversation, we agreed to merge now and support APIs for tagging later if someone asks.
It just occurred to me that we should be totally clear about what is supported and what isn't in the documentation, so someone doesn't try to use e.g. GetObjectTagging and then wonder why it didn't work.
(I'm writing this after you merged the branch so a quick follow-on documentation tweak commit would be appreciated it)
- % Done changed from 0 to 100
- Status changed from In Progress to Resolved
Applied in changeset arvados-private:commit:arvados|607033c33f2001c194fe8c68d0dc17e4bde849da.
Also available in: Atom
PDF
Updated by 
Updated by