https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422018-07-30T15:34:21ZArvadosArvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=650752018-07-30T15:34:21ZJiayong Lijli@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=650762018-07-30T15:37:56ZJiayong Lijli@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/65076/diff?detail_id=61978">diff</a>)</li><li><strong>Status</strong> changed from <i>In Progress</i> to <i>New</i></li></ul> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=650832018-07-30T17:42:34ZJiayong Lijli@curii.com
<ul></ul><p>Related to <a class="external" href="https://projects.veritasgenetics.com/issues/8093">https://projects.veritasgenetics.com/issues/8093</a></p> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=650892018-07-30T20:39:37ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p>This is a bug in a-c-r, it is failing to fill in file sizes for inputs supplied as keep references.</p> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=651942018-08-01T15:46:55ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Assigned To</strong> set to <i>Peter Amstutz</i></li><li><strong>Target version</strong> set to <i>2018-08-15 Sprint</i></li></ul> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=653382018-08-06T21:16:05ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p>13931-cwl-size @ <a class="changeset" title="13931: Add keep size test to test list and fix Dockerfile for dev jobs image Arvados-DCO-1.1-Sig..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/d94ed391d56f17f23e98a8e116849a44cac27e3b">d94ed391d56f17f23e98a8e116849a44cac27e3b</a></p>
<p><a class="external" href="https://ci.curoverse.com/view/Developer/job/developer-run-tests/838/">https://ci.curoverse.com/view/Developer/job/developer-run-tests/838/</a></p>
<p>Corresponding work in cwltool: <a class="external" href="https://github.com/common-workflow-language/cwltool/pull/855">https://github.com/common-workflow-language/cwltool/pull/855</a></p> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=653452018-08-06T21:32:24ZTom Cleggtom@curii.com
<ul></ul><p>Addition LGTM but it looks like various existing tests' expectations need to be updated to accommodate the new size field.</p> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=653832018-08-07T14:11:22ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p>Tom Clegg wrote:</p>
<blockquote>
<p>Addition LGTM but it looks like various existing tests' expectations need to be updated to accommodate the new size field.</p>
</blockquote>
<p>Oops. I guess it is good that the tests are sensitive to that.</p> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=653902018-08-07T14:27:43ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=656602018-08-13T18:45:41ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p>This exposed a different bug with URI quoting filenames:</p>
<pre>
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr Traceback (most recent call last):
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr File "/usr/lib/python2.7/dist-packages/arvados_cwl/crunch_script.py", line 140, in run
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr runner.arv_executor(t, job_order_object, args, logger=logger)
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr File "/usr/lib/python2.7/dist-packages/arvados_cwl/__init__.py", line 559, in arv_executor
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr runtimeContext)
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr File "/usr/lib/python2.7/dist-packages/arvados_cwl/arvtool.py", line 42, in job
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr builder = self._init_job(joborder, builderargs)
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr File "/usr/lib/python2.7/dist-packages/cwltool/process.py", line 594, in _init_job
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr fill_in_defaults(self.tool[u"inputs"], job, fs_access)
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr File "/usr/lib/python2.7/dist-packages/cwltool/process.py", line 392, in fill_in_defaults
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr visit_class(job, ("File",), functools.partial(add_sizes, fsaccess))
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr File "/usr/lib/python2.7/dist-packages/cwltool/utils.py", line 223, in visit_class
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr visit_class(rec[d], cls, op)
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr File "/usr/lib/python2.7/dist-packages/cwltool/utils.py", line 221, in visit_class
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr op(rec)
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr File "/usr/lib/python2.7/dist-packages/cwltool/process.py", line 369, in add_sizes
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr obj["size"] = fsaccess.size(obj["location"])
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr File "/usr/lib/python2.7/dist-packages/arvados_cwl/fsaccess.py", line 149, in size
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr raise IOError(errno.EINVAL, "Not a path to a file %s" % (fn))
2018-08-13_18:36:52 <a href="https://arvadosapi.com/2tlax-8i9sb-vg3zchxctq1q4mk">2tlax-8i9sb-vg3zchxctq1q4mk</a> 29781 0 stderr IOError: [Errno 22] Not a path to a file keep:6bf7c0d0f95d3ca89ce7a1d64ea4ff0a+56/item%20%231.txt
</pre> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=656682018-08-13T20:11:34ZTom Cleggtom@curii.com
<ul></ul><p><a class="changeset" title="13931: un-uriquote keep: URIs when converting to (collection, path) tuple Arvados-DCO-1.1-Signed..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/fc582bfe0561092165f45b7b6f4d0025892aebfd">fc582bfe0</a> LGTM</p> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=656802018-08-14T13:44:09ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=661072018-08-23T15:21:21ZTom Cleggtom@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-3 priority-4 priority-default closed" href="/issues/14074">Bug #14074</a>: a-c-r does not work with input keep locators containing '#' characters</i> added</li></ul> Arvados - Bug #13931: [CWL] size field not accessible on arvadoshttps://dev.arvados.org/issues/13931?journal_id=667922018-09-17T13:18:25ZWard Vandewegeward@curii.com
<ul><li><strong>Release</strong> set to <i>13</i></li></ul>