Story #6830

[API] [keep-balance] Option to return unsigned manifests from collections#index

Added by Radhika Chippada almost 2 years ago. Updated 4 months ago.

Status:ResolvedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:API
Target version:-
Story points-
Velocity based estimate-

Description

In certain use cases (notably keep-balance) it is necessary to retrieve a collection's manifest_text, but not necessary for the locators in the manifest to have valid permission signatures. Currently there is no way for a client to communicate whether signatures are needed, so they are always provided. Therefore, in cases where they aren't needed, much time (as well as memory and network traffic) is wasted by the API server generating signatures that will never be used.

In the case of keep-balance, this wasted time can dominate the runtime of the entire rebalance operation.

Proposed fix:
  • API: Add a response field called "plain_manifest_text" which is unselected by default in both "get" and "index" APIs.
  • API: Ensure the signatures are not included in the response or computed at all when a request selects plain_manifest_text and does not select manifest_text.
  • keep-balance: select the plain_manifest_text field instead of manifest_text.

Related issues

Related to Arvados - Bug #10408: 4xphq keep balance api request causes oom on api server New 10/31/2016
Related to Arvados - Bug #9998: Improve performance of API server when keep-balance retri... Resolved 09/08/2016

History

#1 Updated by Tom Clegg 9 months ago

  • Subject changed from [API] Provide an api call to fetch raw collection records, without signing. This would help reduce the amount of data sent to datamanager. to [API] [keep-balance] Option to return unsigned manifests from collections#index
  • Description updated (diff)
  • Category set to API

#2 Updated by Tom Clegg 4 months ago

  • Status changed from New to Resolved

This was implemented in #9998.

Also available in: Atom PDF