Project

General

Profile

Actions

Bug #8866

closed

[API] 0 items found when requesting a list with filters that aren't URL-encoded

Added by Alexander Afanasyev about 8 years ago. Updated almost 8 years ago.

Status:
Duplicate
Priority:
Normal
Assigned To:
-
Category:
API
Target version:
-
Story points:
-

Description

There are resource's properties referencing to some data by it's portable_data_hash value, e.g. such as log and output properties for Job resource. How can I find out what collections "have this content" (as it does from the Workbench) making an API HTTP calls?
  • If portable_data_hash is used as an uuid of collection while making a call, API will respond with just Keep block locators inforamtion, no information about collections holding this content
  • Search doesn't work with portable_data_hash attribute, e.g. filter condition
    ["portable_data_hash","=","123456a1a11aa1234b1b1112223a1a11+2222"]
    doesn't make any filtration, but this does
    ["portable_data_hash","=",null]

At the moment I'm interested to get Job's log and output collections, and still wondering does Keep know that there are no collections referencing particular content it storing, and if so, what does it do in such cases?


Related issues

Is duplicate of Arvados - Bug #7166: [API] specifying an invalid filter should result in an errorNewActions
Actions #1

Updated by Brett Smith about 8 years ago

Alexander,

Filtering by portable data hash the way you showed should work. For example, if I search for empty collections in our public trial cluster (using my admin account), I see that there are many thanks to the items_available field:

% arv collection list -f '[["portable_data_hash", "=", "d41d8cd98f00b204e9800998ecf8427e+0"]]' --limit=0
{
 "kind":"arvados#collectionList",
 "etag":"",
 "self_link":"",
 "offset":0,
 "limit":0,
 "items":[],
 "items_available":5081
}

Without the filters, I see almost 100,000 collections, so they definitely have an effect.

Are you able to show more specifically what you're doing, in code or a terminal session, that's not working as you expect? That might help us see the problem better.

Actions #2

Updated by Alexander Afanasyev about 8 years ago

Brett, thanks for reply.

Yep, here what I just did (HTTP requests made via Postman to my local Arvados instance):

POST /arvados/v1/collections HTTP/1.1
Host: ...
Authorization: OAuth2 ...
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: ...

{}

Response:

{
  ...
  "portable_data_hash": "d41d8cd98f00b204e9800998ecf8427e+0",
  ...
}

Now I try:

GET /arvados/v1/collections?filters=[["portable_data_hash","=","d41d8cd98f00b204e9800998ecf8427e+0"]] HTTP/1.1
Host: ...
Authorization: OAuth2 ...
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: ...

As a result:

{
  "kind": "arvados#collectionList",
  "etag": "",
  "self_link": "",
  "offset": 0,
  "limit": 100,
  "items": [],
  "items_available": 0
}

Actions #3

Updated by Brett Smith about 8 years ago

Alexander,

Are your filters getting URL-encoded? If that doesn't happen, you get a response as if your filters matched nothing:

% GET -H 'Authorization: OAuth2 ...' 'https://qr1hi.arvadosapi.com/arvados/v1/collections?filters=[["portable_data_hash","=","d41d8cd98f00b204e9800998ecf8427e+0"]]&limit=0'
{"kind":"arvados#collectionList","etag":"","self_link":"","offset":0,"limit":0,"items":[],"items_available":0}

% GET -H 'Authorization: OAuth2 ...' 'https://qr1hi.arvadosapi.com/arvados/v1/collections?filters=%5B%5B%22portable_data_hash%22%2C+%22%3D%22%2C+%22d41d8cd98f00b204e9800998ecf8427e%2B0%22%5D%5D&limit=0'
{"kind":"arvados#collectionList","etag":"","self_link":"","offset":0,"limit":0,"items":[],"items_available":5081}

Does that solve your issue?

Actions #4

Updated by Alexander Afanasyev about 8 years ago

  • Status changed from New to Closed

Yes, it does.
Thank you, Brett!

Actions #5

Updated by Brett Smith about 8 years ago

  • Tracker changed from Support to Bug
  • Subject changed from Search collections by their content to [API] 0 items found when requesting a list with filters that aren't URL-encoded
  • Category set to API
  • Status changed from Closed to New

I'm reopening this as a bug because I feel like we should report a more explicit error when your filters look like this, rather than returning 0 items. We need to dig a little more to figure out how the filters are being interpreted in that case.

Actions #6

Updated by Brett Smith about 8 years ago

  • Target version set to Arvados Future Sprints
Actions #7

Updated by Brett Smith almost 8 years ago

  • Status changed from New to Duplicate

The fundamental bug is the same as #7166.

Actions #8

Updated by Brett Smith almost 8 years ago

  • Target version deleted (Arvados Future Sprints)
Actions

Also available in: Atom PDF