Bug #5560

[DRAFT] [API] Good API for accessing the old_ and new_attributes in Logs table properties column

Added by Tom Clegg about 7 years ago. Updated 10 months ago.

Assigned To:
Target version:
Start date:
Due date:
% Done:


Estimated time:
Story points:


Problems with the old_attributes and new_attributes hashes in the properties hash:
  • They don't look quite close enough to "what the API response to GET would have looked like at the time" for clients to reuse code to interpret/display them (e.g., timestamp formats can be different, computed properties are not present, locators in manifests are not signed).
  • The *_attributes hashes can be huge (notably for collection updates, where there are two copies of the manifest). This uses a lot of database space, and (worse) makes it very slow for clients to retrieve logs (unless they use select to avoid retrieving any properties).
  • Even the associated old_etag and new_etag fields are not indexable or searchable.

Possible approach:

Add a version table (either a single one, or a table per model type), indexed by etag. Store the attributes in the version table. In the logs table, just store the old and new etags. Provide a distinct API for retrieving a specific version of an object (in the usual API response format for that object type) by giving its etag. Optionally, provide an API for retrieving the logs and the object versions referenced by the logs in one request (this would be as complete as the current behavior -- but that isn't necessarily important).

Related issues

Related to Arvados - Bug #5556: [Workbench] Timeout error on collection#show for a collection with large manifest textResolved03/25/2015

Related to Arvados - Story #5554: [API] Delete crunch job log messages from logs table after the jobs finishResolved04/23/2015


#1 Updated by Tom Clegg about 7 years ago

  • Description updated (diff)
  • Category set to API

#2 Updated by Tom Clegg about 7 years ago

  • Target version changed from Arvados Future Sprints to 2015-04-29 sprint

#3 Updated by Tom Clegg about 7 years ago

  • Assigned To set to Tom Clegg

#4 Updated by Tom Clegg about 7 years ago

  • Target version changed from 2015-04-29 sprint to Arvados Future Sprints

#5 Updated by Ward Vandewege 10 months ago

  • Target version deleted (Arvados Future Sprints)

Also available in: Atom PDF