Story #6830

[API] [keep-balance] Option to return unsigned manifests from collections#index

Added by Radhika Chippada over 2 years ago. Updated 9 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
API
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

In certain use cases (notably keep-balance) it is necessary to retrieve a collection's manifest_text, but not necessary for the locators in the manifest to have valid permission signatures. Currently there is no way for a client to communicate whether signatures are needed, so they are always provided. Therefore, in cases where they aren't needed, much time (as well as memory and network traffic) is wasted by the API server generating signatures that will never be used.

In the case of keep-balance, this wasted time can dominate the runtime of the entire rebalance operation.

Proposed fix:
  • API: Add a response field called "plain_manifest_text" which is unselected by default in both "get" and "index" APIs.
  • API: Ensure the signatures are not included in the response or computed at all when a request selects plain_manifest_text and does not select manifest_text.
  • keep-balance: select the plain_manifest_text field instead of manifest_text.

Related issues

Related to Arvados - Bug #10408: 4xphq keep balance api request causes oom on api serverNew2016-10-31

Related to Arvados - Bug #9998: Improve performance of API server when keep-balance retrieves collectionsResolved2016-09-08

History

#1 Updated by Tom Clegg about 1 year ago

  • Subject changed from [API] Provide an api call to fetch raw collection records, without signing. This would help reduce the amount of data sent to datamanager. to [API] [keep-balance] Option to return unsigned manifests from collections#index
  • Description updated (diff)
  • Category set to API

#2 Updated by Tom Clegg 9 months ago

  • Status changed from New to Resolved

This was implemented in #9998.

Also available in: Atom PDF