Feature #9364
[keep-balance] "Expedited delete" tool: perform garbage collection on some specific (recently deleted) collections, bypassing usual GC race protections
Start date:
Due date:
% Done:
0%
Estimated time:
Story points:
3.0
Description
See Expedited delete
Given some PDHes of trashed collections...
For each PDH:- Check for an un-trashed collection with this PDH. If one exists, show a warning: this means "expedited delete" for this collection is a no-op.
- Get the manifest text for this PDH from the trash.
- Don't compute changes for all blocks -- only the ones appearing in the manifests retrieved above.
- When checking trashed collections for recently referenced blocks (see #9363), skip collections with any of the supplied PDHes.
- When deciding whether to delete blocks, use the most recent timestamp of the collections being deleted rather than {now minus signatureTTL} as the race window threshold.
- Use the (new) "synchronous delete, ignoring timestamp" feature of keepstore instead of sending a trash list.
- Don't process any "pull" operations.
- If any blocks in the deleted collections are still referenced by other collections (either trashed or un-trashed), log the PDHes of the collections that prevent the data from being fully deleted.
This could be presented as a separate command ("keep-force-delete"?) if keep-balance is refactored into a module. Alternatively, it could be a runtime option for keep-balance. It will use the same configuration file as keep-balance (along with other options).
Related issues
History
#1
Updated by Tom Clegg over 4 years ago
- Description updated (diff)
#2
Updated by Tom Clegg over 4 years ago
- Description updated (diff)
#3
Updated by Tom Morris about 4 years ago
- Category set to Keep
- Target version set to Arvados Future Sprints
#4
Updated by Tom Clegg almost 4 years ago
- Description updated (diff)
#5
Updated by Tom Clegg almost 4 years ago
- Description updated (diff)
#6
Updated by Tom Clegg almost 4 years ago
- Description updated (diff)
#7
Updated by Tom Clegg almost 4 years ago
- Story points changed from 2.0 to 3.0