Bug #5074: [SDKs] CLI client and Ruby SDK should recover from (and avoid creating) unreadable discovery doc cache files. - Arvados

Actions

Copy link

Bug #5074

closed

[SDKs] CLI client and Ruby SDK should recover from (and avoid creating) unreadable discovery doc cache files.

Added by Tom Clegg about 9 years ago. Updated about 9 years ago.

Status:

Resolved

Priority:

Normal

Assigned To:

Brett Smith

Category:

SDKs

Target version:

2015-01-28 Sprint

Story points:

0.5

Description

The current code creates a race condition while writing a file. It should create a temp file, write it, flush/close it, and rename it into place. If anything fails, it should delete the temp file.

Additionally, if a cache file exists (and is new enough, etc.) but is unreadable or unparseable, the SDK crashes. Instead, it should print a warning, then retrieve the discovery document from the server and continue.

See source:sdk/cli/bin/arv#L55 and source:sdk/ruby/lib/arvados.rb#L108

Bonus points: de-duplicate this copy-and-pasted code!

Subtasks 1 (0 open — 1 closed)

Actions

Copy link

Updated by Tom Clegg about 9 years ago

Subject changed from [SDKs] Ruby SDK should recover from (and avoid creating) unreadable discovery doc cache files. to [SDKs] CLI client and Ruby SDK should recover from (and avoid creating) unreadable discovery doc cache files.
Description updated (diff)
Category set to SDKs

Actions

Copy link

Updated by Brett Smith about 9 years ago

Status changed from New to In Progress
Assigned To set to Brett Smith
Target version changed from Bug Triage to 2015-01-28 Sprint

Actions

Copy link

Updated by Peter Amstutz about 9 years ago

The code looks good. I appreciate that you moved the logic to the Ruby SDK.

So I intentionally corrupted the cached discovery document and got this error:

Warning: error reading cached discovery document /home/peter/.cache/arvados/discovery-86e60ccc4da91c726aa80dfb73fa9cbf.json: 795: 
unexpected token at ':"discovery#restDescription","discoveryVersion":"v1","id":"arvados:v1","name":"arvados","version":"v1","revision":"20131114",
"source_version":"1fae5d5","generatedAt":"2015-01-28T18:52:17+00:00","title":"Arvados API","description":"The API to interact with 
Arvados.","documentationLink":"http://doc.arvados.org/api/index.html","protocol":"rest","baseUrl":"https://qr1hi.arvadosapi.com/arvados/v1/",
... (and so forth for 1000s lines spewing the entire discover doc)

Consider not reporting an error in this case, because it is nonfatal and the user probably can't do anything with the information.

Does the Python discovery document cache code have the same robustness that the Ruby SDK will have with this branch?

Python appears to use a different caching scheme than Ruby, which means the discovery document gets stored twice?

Actions

Copy link

Updated by Brett Smith about 9 years ago

Peter Amstutz wrote:

Consider not reporting an error in this case, because it is nonfatal and the user probably can't do anything with the information.

Yeah, with 100-line error messages that's definitely not going to fly. Updated in d59deccc.

Does the Python discovery document cache code have the same robustness that the Ruby SDK will have with this branch?

The Python SDK lets httplib2 handle caching. I haven't dug into its source but I think we should trust them to deal with it robustly. They have to handle users a lot more demanding than us.

Thanks.

Actions

Copy link