https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422017-03-08T19:32:22ZArvadosArvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491692017-03-08T19:32:22ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/49169/diff?detail_id=47383">diff</a>)</li></ul> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491702017-03-08T19:32:52ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/49170/diff?detail_id=47384">diff</a>)</li></ul> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491712017-03-08T19:41:54ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/49171/diff?detail_id=47385">diff</a>)</li></ul> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491782017-03-09T15:24:30ZTom Cleggtom@curii.com
<ul><li><strong>Subject</strong> changed from <i>[SDK] Not using ~/.cache/arvados/discovery</i> to <i>[SDK] Python client tries to cache discovery document at the same local path regardless of user</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/49178/diff?detail_id=47395">diff</a>)</li></ul> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491792017-03-09T15:26:57ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/49179/diff?detail_id=47396">diff</a>)</li></ul> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491812017-03-09T15:38:08ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p><a class="external" href="https://github.com/google/google-api-python-client/pull/127">https://github.com/google/google-api-python-client/pull/127</a></p>
<p>I think we can use the <code>cache_discovery</code> option to build() to disable the Google API client caching.</p> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491822017-03-09T15:39:17ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/49182/diff?detail_id=47400">diff</a>)</li></ul> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491832017-03-09T15:45:23ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p>Also want to do one of:</p>
<p>a) continue to use httplib2 caching, which is apparently not multithread/multiprocess safe</p>
<p>b) provide our own httplib2 cache which is multithread/multiprocess safe</p>
<p>c) provide a google api client cache object (with <code>build(cache=XXX)</code>) which is multithread/multiprocess safe:<br /><a class="external" href="https://github.com/google/google-api-python-client/blob/v1.4.2/googleapiclient/discovery_cache/base.py">https://github.com/google/google-api-python-client/blob/v1.4.2/googleapiclient/discovery_cache/base.py</a></p> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491842017-03-09T15:48:25ZTom Cleggtom@curii.com
<ul></ul>Suggest we
<ol>
<li>pass cache_discovery=False to google-api-client so we get back to what we thought we were doing, i.e., sensible http caching</li>
<li>also fix <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: googleapiclient.errors.InvalidJsonError retrieving discovery document (Resolved)" href="https://dev.arvados.org/issues/10669">#10669</a> so the http cache is safe for multiple processes to use</li>
</ol> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491852017-03-09T15:49:28ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p>+1</p> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491862017-03-09T15:51:28ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Assigned To</strong> set to <i>Peter Amstutz</i></li><li><strong>Story points</strong> set to <i>0.5</i></li></ul> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491872017-03-09T15:54:46ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Target version</strong> set to <i>2017-03-15 sprint</i></li></ul> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=491932017-03-09T19:50:05ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset arvados|commit:82697fea93b1c87cdee27d2b9a76c1b7ac07497e.</p> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=494002017-03-13T16:19:25ZTom Morristfmorris@veritasgenetics.com
<ul></ul><p>I take it this got instagroomed while I was gone.</p>
<p>Does this mean that <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: googleapiclient.errors.InvalidJsonError retrieving discovery document (Resolved)" href="https://dev.arvados.org/issues/10669">#10669</a> is fixed (and can be closed) as well?</p> Arvados - Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardless of userhttps://dev.arvados.org/issues/11226?journal_id=494012017-03-13T16:55:31ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p>Tom Morris wrote:</p>
<blockquote>
<p>I take it this got instagroomed while I was gone.</p>
<p>Does this mean that <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: googleapiclient.errors.InvalidJsonError retrieving discovery document (Resolved)" href="https://dev.arvados.org/issues/10669">#10669</a> is fixed (and can be closed) as well?</p>
</blockquote>
<p>No, this only disables google-api-client caching so that it uses the httplib2 cache as originally intended. The bug in <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: googleapiclient.errors.InvalidJsonError retrieving discovery document (Resolved)" href="https://dev.arvados.org/issues/10669">#10669</a> is that the httplib2 cache is not multi-thread/multi-process safe, and which makes it possible to get corrupted. The google-api-client cache actually does do file locking, but puts the cache file in the global <code>/tmp</code> and doesn't provide a way to change that except for changing the global tempfile path (which has side effects for everything using the tempfile module).</p>