Arvados

h1. Collection update API

Problem: adding, updating, renaming, or removing a file in a collection requires sending the entire manifest text of the collection.  This approach creates two problems:

# It is inefficient; resources are wasted transferring and processing manifest data that hasn't changed 

# It creates a race condition; for example processes A and B read the collection record at time T1, merge it with their local updates, process A sends its update at time T2, process B sends its update at time T3, then in process A's updates are lost.

Proposed solution: introduce new 'update' or 'patch' endpoint for collections supporting a patching protocol allowing incremental collection update.

The protocol consists of an *operation* field with values of *add*, *remove* or *modify*.

Each operation has parameters *initial_contents*, and *final_contents*.  The *contents* fields contain normalized manifests with exactly one file.  The stream and file name are put together to get the name of the file that is affected by the operation.

An *add* operation adds a new file represented by *final_contents*.  The *initial_contents* should be empty or unspecified.  If there is already a file with that name and the existing file has different contents, the existing file should be uniquely renamed to indicate that it is in conflict, and the new file takes its place.

An *remove* operations adds