Project

General

Profile

Actions

Bug #21205

closed

ensure_unique_name logic is not robust enough

Added by Peter Amstutz 3 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
API
Start date:
11/29/2023
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

User reported "duplicate key value violates unique constraint index_collections_on_owner_and_name".

This error occurred from arvados-cwl-runner creating a collection called "Intermediate collection".

This case does supply the ensure_unique_name flag.

However, it seems that between running 100s of workflows, and this particular step doing a 24-way scatter, that it attempting to create ~10 collections per second (see screenshot), and that, while < 1% chance, it is evidently possible for two or more collections to be created at the same time and for save_with_unique_name! to fail enough times in a row that one of those collection create operations fails with a uniqueness error.

From discussion:

Instead of adding a timestamp, we should add ~10 random alphanumeric digits (similar to the logic used for UUIDs). As demonstrated, timestamps are prone to collision, and using millisecond precision that means we are limited to creating 1000 collections per second.


Files


Subtasks 1 (0 open1 closed)

Task #21208: Review 21205-ensure-uniqueResolvedPeter Amstutz11/29/2023

Actions
Actions

Also available in: Atom PDF