Story #16427

"undelete" command to recover trashed blocks and restore a deleted collection

Added by Tom Clegg 21 days ago. Updated 1 day ago.

Status:
In Progress
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
06/01/2020
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Story points:
3.0

Description

In some cases, even when a collection has been deleted and keep-balance has trashed its data blocks, the collection is still partially or fully recoverable:
  • some blocks may not have been trashed because they are still referenced by other collections
  • some blocks may still be in the recoverable "trashed" state, if BlobTrashLifetime has not arrived yet

Given a manifest (or the UUID of a a collection update/delete log entry that has a manifest in old_attributes), the recovery command ("arvados-server undelete") should use keepstore's HEAD and untrash APIs to untrash the data as needed, make a new manifest with fresh signatures, and save a new collection.

If a block is not recoverable, the command should continue to untrash as many as it can, and report how many were attempted/successful, but not save a new collection. (Future improvement: save a partial collection in this case, omitting any files affected by the missing blocks.)


Subtasks

Task #16454: Review 16427-undeleteIn ProgressWard Vandewege


Related issues

Related to Arvados - Support #16421: [doc] document deletion lifecycle of collections, and steps to undelete collectionsNew

History

#1 Updated by Tom Clegg 20 days ago

  • Story points set to 3.0

#2 Updated by Peter Amstutz 20 days ago

  • Target version set to 2020-06-03 Sprint

#4 Updated by Tom Clegg 19 days ago

  • Description updated (diff)

#5 Updated by Tom Clegg 18 days ago

  • Status changed from New to In Progress

work in progress at 16427-undelete

#6 Updated by Peter Amstutz 13 days ago

  • Assigned To set to Tom Clegg

#7 Updated by Ward Vandewege 11 days ago

  • Related to Support #16421: [doc] document deletion lifecycle of collections, and steps to undelete collections added

#8 Updated by Tom Clegg 4 days ago

16427-undelete @ 167e2537365ec68fe6be6a330a2eb698f177aa05 -- https://ci.arvados.org/view/Developer/job/developer-run-tests/1879/

TODO: accept UUID of collection-delete/update log entry (currently must provide file containing manifest text)
TODO: doc page

Also available in: Atom PDF