Arvados

h1. Collection version history

(draft)

Background: desired API features (#13494, #13561) epic ticket is #13109

* Get current + previous versions of a collection with a single API call

* Search all current + previous versions of collections with a single API call

* Stable small numeric "version number" for each version of a given collection

h2. Design

* Store all versions in the collection table (much easier to do paging)

* New database column "current_version_uuid" for latest version this the same as "uuid"

* New database column indicates user-friendly version number (starting at 1) -- can be assigned safely after locking "current" row for update

* Assign version number to the current version -- this way {uuid,version} can be used as a permanent reference to a specific version

* Flag "include_past_versions" for "list" API to include old versions (otherwise, filter on current_version_uuid=uuid -- might need to adjust indexes to maintain performance)

* Flag in site configuration to enable preserving old versions in database

* Update owner_uuid of all old versions (in a transaction) when it changes in the current version

* A permission link to the current version only implies permission to the most recent version (not the history)

* Prohibit permission links to old versions (should be handled by check "can only make links to things that are visible to you", which should not include old versions)

* Old versions may not be modified except for certain fields which must be synchronized with "current" version changes (owner_uuid, trash_at, delete_at).  Updating owner_uuid, trash_at, delete_at does not update modified_at so modified_at shows when the version was created

* Old versions with duplicate "name" does not conflict with (owner_uuid, name) uniqueness constraint

* The following changes introduce a new version: manifest_text (portable_data_hash), description, properties, name, 

* The following changes do not introduce a new version, and a copied to all past version records: replication_*, storage_classes_*, trash_at/delete_at/is_trashed, owner_uuid, uuid (update current_uuid to retain database consistency)

h2. Implementation

* Implement "Collection update" in arvados-controller (Go)

** Intercept update operations only (PATCH, POST _method=PATCH?)

** Check valid manifest

** Check valid signatures

** Strip signatures

** Check/update PDH

** Update file_names

** Copy old version to new PDH

** Update version number

** Write record

** Return record (this means GET effectively has to be implemented as well?)

* Configuration option to enable/disable "update" on arvados-controller and reeanble on API server as fallback