Project

General

Profile

Actions

Bug #8910

closed

[SDK] arv-put should save manifest text on API error

Added by Peter Amstutz about 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

If arv-put gets an API error when creating the collection record, the entire upload is for nothing because the once the manifest_text is lost it is necessary to re-upload the whole thing. Arv-put should save the manifest text somewhere to offer some way of recovering from error.

This should probably leverage the existing "resume" functionality.


Related issues

Related to Arvados - Bug #8878: Keep: sudden appearance of "missing" blocksClosed04/04/2016

Actions
Blocked by Arvados - Story #9463: [SDKs] Change arv-put to use the Collection class under the hoodResolvedLucas Di Pentima07/11/2016

Actions
Actions #1

Updated by Peter Amstutz about 6 years ago

  • Description updated (diff)
Actions #2

Updated by Brett Smith about 6 years ago

Peter Amstutz wrote:

This should probably leverage the existing "resume" functionality.

Another approach would be for arv-put to start using the Collection class, start a new collection on upload, and periodically save it until the upload is finished.

This is a bigger change, but moving to the Collection class makes it easier to make a lot of other changes we want in arv-put: refreshing permission tokens as it goes along, smarter block splits, adding files to existing collections, etc.

Actions #3

Updated by Brett Smith about 6 years ago

  • Target version set to Arvados Future Sprints
Actions #4

Updated by Tom Morris over 5 years ago

  • Assigned To set to Lucas Di Pentima

Lucas - have the recent arv-put changes resolved this?

Actions #5

Updated by Lucas Di Pentima over 5 years ago

Tom Morris wrote:

Lucas - have the recent arv-put changes resolved this?

Now arv-put periodically (every 20 secs, IIRC) saves the manifest on its cache, so I would say: yes.

Actions #6

Updated by Lucas Di Pentima over 5 years ago

  • Status changed from New to Resolved
Actions #7

Updated by Tom Clegg over 5 years ago

  • Target version deleted (Arvados Future Sprints)
Actions

Also available in: Atom PDF