Bug #5560


[DRAFT] [API] Good API for accessing the old_ and new_attributes in Logs table properties column

Added by Tom Clegg over 7 years ago. Updated about 1 year ago.

Assigned To:
Target version:
Start date:
Due date:
% Done:


Estimated time:
Story points:


Problems with the old_attributes and new_attributes hashes in the properties hash:
  • They don't look quite close enough to "what the API response to GET would have looked like at the time" for clients to reuse code to interpret/display them (e.g., timestamp formats can be different, computed properties are not present, locators in manifests are not signed).
  • The *_attributes hashes can be huge (notably for collection updates, where there are two copies of the manifest). This uses a lot of database space, and (worse) makes it very slow for clients to retrieve logs (unless they use select to avoid retrieving any properties).
  • Even the associated old_etag and new_etag fields are not indexable or searchable.

Possible approach:

Add a version table (either a single one, or a table per model type), indexed by etag. Store the attributes in the version table. In the logs table, just store the old and new etags. Provide a distinct API for retrieving a specific version of an object (in the usual API response format for that object type) by giving its etag. Optionally, provide an API for retrieving the logs and the object versions referenced by the logs in one request (this would be as complete as the current behavior -- but that isn't necessarily important).

Related issues

Related to Arvados - Bug #5556: [Workbench] Timeout error on collection#show for a collection with large manifest textResolvedRadhika Chippada03/25/2015

Related to Arvados - Story #5554: [API] Delete crunch job log messages from logs table after the jobs finishResolvedBrett Smith04/23/2015

Actions #1

Updated by Tom Clegg over 7 years ago

  • Description updated (diff)
  • Category set to API
Actions #2

Updated by Tom Clegg over 7 years ago

  • Target version changed from Arvados Future Sprints to 2015-04-29 sprint
Actions #3

Updated by Tom Clegg over 7 years ago

  • Assigned To set to Tom Clegg
Actions #4

Updated by Tom Clegg over 7 years ago

  • Target version changed from 2015-04-29 sprint to Arvados Future Sprints
Actions #5

Updated by Ward Vandewege about 1 year ago

  • Target version deleted (Arvados Future Sprints)

Also available in: Atom PDF