Project

General

Profile

Actions

Idea #20693

closed

Design for server side coordination of multiple writers to a collection

Added by Peter Amstutz about 1 year ago. Updated 16 days ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Keep
Start date:
05/24/2024
Due date:
Story points:
-

Description

Background:

Multiple Arvados services (multiple instances of keep-web, arvados-client mount, arv-mount, etc) are trying to write files to the same collection at the same time.

Assume they are adding/removing/changing multiple files but not making changes that directly conflict/contradict one another.

Requirements:

  • If a file is created or modified it won't disappear as a result of an update from another service that didn't know about that file
  • If there is a single writer, performance impact should be minimal
  • If there are multiple writers, it is acceptable that one of them may have to wait to avoid conflicts
  • Can use pessimistic locking to ensure only one client can have a write lock at a time, attempting to open a file for writing that is locked by another should return an error on open
  • Support WebDAV lock protocol

Proposal: Concurrent writes to a single collection


Subtasks 1 (0 open1 closed)

Task #21625: Review at engineering meetingResolved05/24/2024Actions

Related issues

Related to Arvados Epics - Idea #18342: Keep performance optimizationNew08/01/202305/30/2024Actions
Actions

Also available in: Atom PDF