Project

General

Profile

Actions

Bug #6076

open

[API] walk api server installations and ensure modified_at for collections is unique + ensure modified_at is enforced to be unique at the api level.

Added by Ward Vandewege almost 9 years ago. Updated 28 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Story points:
-
Release:
Release relationship:
Auto

Description

Until this is done, datamanager needs to ask for groups of collections with overlap. That is not the most efficient, but it is also potentially problematic in light of #5834 which dynamically adjust the result set based on result size (and only guarantees a minimum of 1 result).


Related issues

Related to Arvados - Idea #3408: [Keep] Implement Production Data ManagerResolvedPeter Amstutz07/29/2014Actions
Actions #1

Updated by Ward Vandewege almost 9 years ago

  • Subject changed from [API] walk api server installations and ensure modified_at for collections is unique to [API] walk api server installations and ensure modified_at for collections is unique + ensure modified_at is unique at the api level.
  • Description updated (diff)
Actions #2

Updated by Ward Vandewege almost 9 years ago

  • Subject changed from [API] walk api server installations and ensure modified_at for collections is unique + ensure modified_at is unique at the api level. to [API] walk api server installations and ensure modified_at for collections is unique + ensure modified_at is enforced to be unique at the api level.
Actions #3

Updated by Brett Smith almost 9 years ago

Isn't this easy enough to handle at the client level with filters like modified_at >= current_working_modified_time, uuid not in [list of UUIDs I've already seen with that modified time]?

Actions #4

Updated by Tom Clegg almost 9 years ago

The goal here is to provide a safe and easy way for a client to get the next page of results. Note-3 is safe but not easy. Ensuring modified_at uniqueness helps make the "modified_at >= X" method safe, but even that is a bit awkward.

In order for "sort by modified_at" to be completely safe, we must guarantee at the database level that modified_at is not only unique, but greater than all other rows' modified_at values at the time of insert/update. Currently, modified_at is assigned by Rails model validations and (AFAIK) there is nothing in place to prevent updates from getting reordered between "assign modified_at" and "insert/update database record". I think it's a good idea to enforce this constraint.

An even better solution would be for the server to provide a "next page token" which the client can send with its next request. This way the "next page" logic can sit on the server side instead of the client side, and the client doesn't need to know exactly which ordering guarantees are enforced. It can stick to "if the server doesn't give me a safe next-page link, I can't work."

Actions #5

Updated by Brett Smith over 8 years ago

  • Target version changed from Bug Triage to Arvados Future Sprints
Actions #6

Updated by Ward Vandewege over 2 years ago

  • Target version deleted (Arvados Future Sprints)
Actions #7

Updated by Peter Amstutz about 1 year ago

  • Release set to 60
Actions #8

Updated by Peter Amstutz 28 days ago

  • Target version set to Future
Actions

Also available in: Atom PDF