Bug #8866
closed[API] 0 items found when requesting a list with filters that aren't URL-encoded
Description
- If portable_data_hash is used as an uuid of collection while making a call, API will respond with just Keep block locators inforamtion, no information about collections holding this content
- Search doesn't work with portable_data_hash attribute, e.g. filter condition
["portable_data_hash","=","123456a1a11aa1234b1b1112223a1a11+2222"]
doesn't make any filtration, but this does["portable_data_hash","=",null]
At the moment I'm interested to get Job's log and output collections, and still wondering does Keep know that there are no collections referencing particular content it storing, and if so, what does it do in such cases?
Updated by Brett Smith over 8 years ago
Alexander,
Filtering by portable data hash the way you showed should work. For example, if I search for empty collections in our public trial cluster (using my admin account), I see that there are many thanks to the items_available field:
% arv collection list -f '[["portable_data_hash", "=", "d41d8cd98f00b204e9800998ecf8427e+0"]]' --limit=0 { "kind":"arvados#collectionList", "etag":"", "self_link":"", "offset":0, "limit":0, "items":[], "items_available":5081 }
Without the filters, I see almost 100,000 collections, so they definitely have an effect.
Are you able to show more specifically what you're doing, in code or a terminal session, that's not working as you expect? That might help us see the problem better.
Updated by Alexander Afanasyev over 8 years ago
Brett, thanks for reply.
Yep, here what I just did (HTTP requests made via Postman to my local Arvados instance):
POST /arvados/v1/collections HTTP/1.1 Host: ... Authorization: OAuth2 ... Content-Type: application/json Cache-Control: no-cache Postman-Token: ... {}
Response:
{ ... "portable_data_hash": "d41d8cd98f00b204e9800998ecf8427e+0", ... }
Now I try:
GET /arvados/v1/collections?filters=[["portable_data_hash","=","d41d8cd98f00b204e9800998ecf8427e+0"]] HTTP/1.1 Host: ... Authorization: OAuth2 ... Content-Type: application/json Cache-Control: no-cache Postman-Token: ...
As a result:
{ "kind": "arvados#collectionList", "etag": "", "self_link": "", "offset": 0, "limit": 100, "items": [], "items_available": 0 }
Updated by Brett Smith over 8 years ago
Alexander,
Are your filters getting URL-encoded? If that doesn't happen, you get a response as if your filters matched nothing:
% GET -H 'Authorization: OAuth2 ...' 'https://qr1hi.arvadosapi.com/arvados/v1/collections?filters=[["portable_data_hash","=","d41d8cd98f00b204e9800998ecf8427e+0"]]&limit=0' {"kind":"arvados#collectionList","etag":"","self_link":"","offset":0,"limit":0,"items":[],"items_available":0} % GET -H 'Authorization: OAuth2 ...' 'https://qr1hi.arvadosapi.com/arvados/v1/collections?filters=%5B%5B%22portable_data_hash%22%2C+%22%3D%22%2C+%22d41d8cd98f00b204e9800998ecf8427e%2B0%22%5D%5D&limit=0' {"kind":"arvados#collectionList","etag":"","self_link":"","offset":0,"limit":0,"items":[],"items_available":5081}
Does that solve your issue?
Updated by Alexander Afanasyev over 8 years ago
- Status changed from New to Closed
Yes, it does.
Thank you, Brett!
Updated by Brett Smith over 8 years ago
- Tracker changed from Support to Bug
- Subject changed from Search collections by their content to [API] 0 items found when requesting a list with filters that aren't URL-encoded
- Category set to API
- Status changed from Closed to New
I'm reopening this as a bug because I feel like we should report a more explicit error when your filters look like this, rather than returning 0 items. We need to dig a little more to figure out how the filters are being interpreted in that case.
Updated by Brett Smith over 8 years ago
- Target version set to Arvados Future Sprints
Updated by Brett Smith over 8 years ago
- Status changed from New to Duplicate
The fundamental bug is the same as #7166.
Updated by Brett Smith over 8 years ago
- Target version deleted (
Arvados Future Sprints)