Actions
Feature #21127
open
Keepstore uses "DeleteObjects" to delete multiple S3 objects at a time to improve throughput
Story points:
-
Description
Deleting 1000s of objects one at a time is time consuming and runs into S3 API limits.
S3 offers a "DeleteObjects" API to provide 1000 objects to delete at a time. We should use this in the Trash() and EmptyTrash() keepstore operations to delete objects faster.
This requires converting these methods into batch versions that take an array of objects.
Updated by Peter Amstutz about 1 year ago
- Description updated (diff)
- Subject changed from Use "DeleteObjects" to delete up to 1000 S3 objects at a time to improve throughput to Keepstore uses "DeleteObjects" to delete up to 1000 S3 objects at a time to improve throughput
Updated by Peter Amstutz about 1 year ago
- Description updated (diff)
- Subject changed from Keepstore uses "DeleteObjects" to delete up to 1000 S3 objects at a time to improve throughput to Keepstore uses "DeleteObjects" to delete multiple S3 objects at a time to improve throughput
Updated by Peter Amstutz about 1 year ago
- Target version changed from Development 2023-11-08 sprint to To be scheduled
Updated by Peter Amstutz 10 months ago
- Target version changed from To be scheduled to Future
Actions