Project

General

Profile

Actions

Feature #20693

open

Server side coordination of multiple writers to a collection

Added by Peter Amstutz 5 months ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Keep
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Background:

Multiple Arvados services (multiple instances of keep-web, arvados-client mount, arv-mount, etc) are trying to write files to the same collection at the same time.

Assume they are adding/removing/changing multiple files but not making changes that directly conflict/contradict one another.

Requirements:

  • If a file is created or modified it won't disappear as a result of an update from another service that didn't know about that file
  • If there is a single writer, performance impact should be minimal
  • If there are multiple writers, it is acceptable that one of them may have to wait to avoid conflicts
  • Can use pessimistic locking to ensure only one client can have a write lock at a time, attempting to open a file for writing that is locked by another should return an error on open
  • Support WebDAV lock protocol

Related issues

Related to Arvados Epics - Story #18342: Keep performance optimizationNew08/01/202310/31/2023

Actions
Actions #1

Updated by Peter Amstutz 5 months ago

  • Status changed from New to In Progress
Actions #2

Updated by Peter Amstutz 5 months ago

  • Status changed from In Progress to New
  • Tracker changed from Bug to Feature
Actions #3

Updated by Peter Amstutz 5 months ago

  • Category set to Keep
Actions #4

Updated by Peter Amstutz 5 months ago

  • Description updated (diff)
Actions #5

Updated by Peter Amstutz 5 months ago

  • Related to Story #18342: Keep performance optimization added
Actions #6

Updated by Peter Amstutz 2 months ago

  • Target version changed from To be groomed to Development 2024-01-17 sprint
Actions #7

Updated by Peter Amstutz 2 months ago

  • Target version changed from Development 2024-01-17 sprint to Development 2024-01-03 sprint
Actions #8

Updated by Peter Amstutz about 1 month ago

  • Target version changed from Development 2024-01-03 sprint to Development 2024-01-17 sprint
Actions

Also available in: Atom PDF