Feature #21127
open
Keepstore uses "DeleteObjects" to delete multiple S3 objects at a time to improve throughput
Added by Peter Amstutz 7 months ago.
Updated 2 months ago.
Description
Deleting 1000s of objects one at a time is time consuming and runs into S3 API limits.
S3 offers a "DeleteObjects" API to provide 1000 objects to delete at a time. We should use this in the Trash() and EmptyTrash() keepstore operations to delete objects faster.
This requires converting these methods into batch versions that take an array of objects.
- Description updated (diff)
- Subject changed from Use "DeleteObjects" to delete up to 1000 S3 objects at a time to improve throughput to Keepstore uses "DeleteObjects" to delete up to 1000 S3 objects at a time to improve throughput
- Description updated (diff)
- Subject changed from Keepstore uses "DeleteObjects" to delete up to 1000 S3 objects at a time to improve throughput to Keepstore uses "DeleteObjects" to delete multiple S3 objects at a time to improve throughput
- Target version changed from Development 2023-11-08 sprint to To be scheduled
- Target version changed from To be scheduled to Future
Also available in: Atom
PDF
Updated by