https://dev.arvados.org/
https://dev.arvados.org/favicon.ico?1557688842
2016-11-21T15:07:01Z
Arvados
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=45507
2016-11-21T15:07:01Z
Peter Amstutz
peter.amstutz@curii.com
<ul><li><strong>Subject</strong> changed from <i>[CWL] Link checking is able to recognize keep references</i> to <i>[CWL] Link checking must be able to recognize keep references</i></li></ul>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=45512
2016-11-21T15:59:29Z
Tom Morris
tfmorris@veritasgenetics.com
<ul><li><strong>Assigned To</strong> set to <i>Peter Amstutz</i></li><li><strong>Target version</strong> set to <i>2016-11-23 sprint</i></li></ul>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=45682
2016-11-23T19:37:12Z
Peter Amstutz
peter.amstutz@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>Target version</strong> changed from <i>2016-11-23 sprint</i> to <i>2016-12-14 sprint</i></li></ul>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=45713
2016-11-23T19:53:58Z
Peter Amstutz
peter.amstutz@curii.com
<ul><li><strong>Story points</strong> set to <i>2.0</i></li></ul>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=45714
2016-11-23T19:54:14Z
Peter Amstutz
peter.amstutz@curii.com
<ul><li><strong>Story points</strong> changed from <i>2.0</i> to <i>1.0</i></li></ul>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=46140
2016-12-07T18:41:46Z
Radhika Chippada
radhika@curoverse.com
<ul></ul><p>Still reviewing, but an initial observation: 7 tests failing (after a reinstall)</p>
<pre>
======================================================================
FAIL: test_submit_arvworkflow (tests.test_submit.TestSubmit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 213, in wrapped
return func(self, stubs, *args, **kwargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 636, in test_submit_arvworkflow
body=expect_container)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 937, in assert_called_with
six.raise_from(AssertionError(_error_message(cause)), cause)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/six.py", line 718, in raise_from
raise value
AssertionError: Expected call: create(body={'priority': 1, 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'state': 'Committed', 'owner_uuid': None, 'output_path': '/var/spool/cwl', 'name': 'arvwf:<a href="https://arvadosapi.com/962eh-7fd4e-gkbzl62qqtfig37">962eh-7fd4e-gkbzl62qqtfig37</a>#main', 'mounts': {'/var/lib/cwl/workflow.json': {'kind': 'json', 'json': {'cwlVersion': 'v1.0', '$graph': [{'inputs': [{'inputBinding': {'position': 1}, 'type': 'string', 'id': '#submit_tool.cwl/x'}], 'requirements': [{'dockerPull': 'debian:8', 'class': 'DockerRequirement'}], 'outputs': [], 'baseCommand': 'cat', 'id': '#submit_tool.cwl', 'class': 'CommandLineTool'}, {'inputs': [{'type': 'string', 'id': '#main/x'}], 'steps': [{'out': [], 'run': '#submit_tool.cwl', 'id': '#main/step1', 'in': [{'source': '#main/x', 'id': '#main/step1/x'}]}], 'id': '#main', 'outputs': [], 'class': 'Workflow'}]}}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}, '/var/lib/cwl/job/cwl.input.json': {'portable_data_hash': 'e5454f8ca7d5b181e21ecd45841e3373+58/cwl.input.json', 'kind': 'collection'}, '/var/spool/cwl': {'writable': True, 'kind': 'collection'}}, 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--enable-reuse', '/var/lib/cwl/workflow.json#main', '/var/lib/cwl/job/cwl.input.json'], 'cwd': '/var/spool/cwl', 'runtime_constraints': {'vcpus': 1, 'API': True, 'ram': 1073741824}})
Actual call: create(body={'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}, '/var/lib/cwl/workflow.json': {'kind': 'json', 'json': {'cwlVersion': 'v1.0', '$graph': [{'inputs': [{'inputBinding': {'position': 1}, 'type': 'string', 'id': '#submit_tool.cwl/x'}], 'requirements': [{'dockerPull': 'debian:8', 'class': 'DockerRequirement'}], 'id': '#submit_tool.cwl', 'outputs': [], 'baseCommand': 'cat', 'class': 'CommandLineTool'}, {'id': '#main', 'inputs': [{'type': 'string', 'id': '#main/x'}], 'steps': [{'in': [{'source': '#main/x', 'id': '#main/step1/x'}], 'run': '#submit_tool.cwl', 'id': '#main/step1', 'out': []}], 'class': 'Workflow', 'outputs': []}]}}, '/var/lib/cwl/cwl.input.json': {'content': {u'x': u'XxX', 'job_order': None}, 'kind': 'json'}}, 'state': 'Committed', 'owner_uuid': None, 'output_path': '/var/spool/cwl', 'name': 'arvwf:<a href="https://arvadosapi.com/962eh-7fd4e-gkbzl62qqtfig37">962eh-7fd4e-gkbzl62qqtfig37</a>#main', 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--enable-reuse', '/var/lib/cwl/workflow.json#main', '/var/lib/cwl/cwl.input.json'], 'cwd': '/var/spool/cwl', 'runtime_constraints': {'API': True, 'vcpus': 1, 'ram': 1073741824}})
======================================================================
FAIL: test_submit_container (tests.test_submit.TestSubmit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 213, in wrapped
return func(self, stubs, *args, **kwargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 404, in test_submit_container
body=expect_container)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 937, in assert_called_with
six.raise_from(AssertionError(_error_message(cause)), cause)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/six.py", line 718, in raise_from
raise value
AssertionError: Expected call: create(body={'name': 'submit_wf.cwl', 'runtime_constraints': {'vcpus': 1, 'API': True, 'ram': 1073741824}, 'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/job/cwl.input.json': {'portable_data_hash': 'd20d7cddd1984f105dd3702c7f125afb+60/cwl.input.json', 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}}, 'state': 'Committed', 'owner_uuid': None, 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--enable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/job/cwl.input.json'], 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'output_path': '/var/spool/cwl', 'cwd': '/var/spool/cwl'})
Actual call: create(body={'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/cwl.input.json': {'content': {'y': {'basename': '99999999999999999999999999999998+99', 'location': 'keep:99999999999999999999999999999998+99', 'class': 'Directory'}, 'x': {'basename': u'blorp.txt', 'class': 'File', 'location': u'keep:99999999999999999999999999999994+99/blorp.txt'}, 'z': {'basename': 'anonymous', 'class': 'Directory', 'listing': [{'basename': 'renamed.txt', 'class': 'File', 'location': 'keep:99999999999999999999999999999998+99/file1.txt'}]}}, 'kind': 'json'}}, 'state': 'Committed', 'owner_uuid': None, 'output_path': '/var/spool/cwl', 'name': 'submit_wf.cwl', 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--enable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/cwl.input.json'], 'cwd': '/var/spool/cwl', 'runtime_constraints': {'API': True, 'vcpus': 1, 'ram': 1073741824}})
======================================================================
FAIL: test_submit_container_no_reuse (tests.test_submit.TestSubmit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 213, in wrapped
return func(self, stubs, *args, **kwargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 424, in test_submit_container_no_reuse
body=expect_container)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 937, in assert_called_with
six.raise_from(AssertionError(_error_message(cause)), cause)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/six.py", line 718, in raise_from
raise value
AssertionError: Expected call: create(body={'name': 'submit_wf.cwl', 'runtime_constraints': {'vcpus': 1, 'API': True, 'ram': 1073741824}, 'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/job/cwl.input.json': {'portable_data_hash': 'd20d7cddd1984f105dd3702c7f125afb+60/cwl.input.json', 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}}, 'state': 'Committed', 'owner_uuid': None, 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--disable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/job/cwl.input.json'], 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'output_path': '/var/spool/cwl', 'cwd': '/var/spool/cwl'})
Actual call: create(body={'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/cwl.input.json': {'content': {'y': {'basename': '99999999999999999999999999999998+99', 'location': 'keep:99999999999999999999999999999998+99', 'class': 'Directory'}, 'x': {'basename': u'blorp.txt', 'class': 'File', 'location': u'keep:99999999999999999999999999999994+99/blorp.txt'}, 'z': {'basename': 'anonymous', 'class': 'Directory', 'listing': [{'basename': 'renamed.txt', 'class': 'File', 'location': 'keep:99999999999999999999999999999998+99/file1.txt'}]}}, 'kind': 'json'}}, 'state': 'Committed', 'owner_uuid': None, 'output_path': '/var/spool/cwl', 'name': 'submit_wf.cwl', 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--disable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/cwl.input.json'], 'cwd': '/var/spool/cwl', 'runtime_constraints': {'API': True, 'vcpus': 1, 'ram': 1073741824}})
======================================================================
FAIL: test_submit_container_output_name (tests.test_submit.TestSubmit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 213, in wrapped
return func(self, stubs, *args, **kwargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 446, in test_submit_container_output_name
body=expect_container)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 937, in assert_called_with
six.raise_from(AssertionError(_error_message(cause)), cause)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/six.py", line 718, in raise_from
raise value
AssertionError: Expected call: create(body={'name': 'submit_wf.cwl', 'runtime_constraints': {'vcpus': 1, 'API': True, 'ram': 1073741824}, 'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/job/cwl.input.json': {'portable_data_hash': 'd20d7cddd1984f105dd3702c7f125afb+60/cwl.input.json', 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}}, 'state': 'Committed', 'owner_uuid': None, 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--output-name=test_output_name', '--enable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/job/cwl.input.json'], 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'output_path': '/var/spool/cwl', 'cwd': '/var/spool/cwl'})
Actual call: create(body={'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/cwl.input.json': {'content': {'y': {'basename': '99999999999999999999999999999998+99', 'location': 'keep:99999999999999999999999999999998+99', 'class': 'Directory'}, 'x': {'basename': u'blorp.txt', 'class': 'File', 'location': u'keep:99999999999999999999999999999994+99/blorp.txt'}, 'z': {'basename': 'anonymous', 'class': 'Directory', 'listing': [{'basename': 'renamed.txt', 'class': 'File', 'location': 'keep:99999999999999999999999999999998+99/file1.txt'}]}}, 'kind': 'json'}}, 'state': 'Committed', 'owner_uuid': None, 'output_path': '/var/spool/cwl', 'name': 'submit_wf.cwl', 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--output-name=test_output_name', '--enable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/cwl.input.json'], 'cwd': '/var/spool/cwl', 'runtime_constraints': {'API': True, 'vcpus': 1, 'ram': 1073741824}})
======================================================================
FAIL: test_submit_container_output_tags (tests.test_submit.TestSubmit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 213, in wrapped
return func(self, stubs, *args, **kwargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 468, in test_submit_container_output_tags
body=expect_container)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 937, in assert_called_with
six.raise_from(AssertionError(_error_message(cause)), cause)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/six.py", line 718, in raise_from
raise value
AssertionError: Expected call: create(body={'name': 'submit_wf.cwl', 'runtime_constraints': {'vcpus': 1, 'API': True, 'ram': 1073741824}, 'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/job/cwl.input.json': {'portable_data_hash': 'd20d7cddd1984f105dd3702c7f125afb+60/cwl.input.json', 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}}, 'state': 'Committed', 'owner_uuid': None, 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--output-tags=tag0,tag1,tag2', '--enable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/job/cwl.input.json'], 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'output_path': '/var/spool/cwl', 'cwd': '/var/spool/cwl'})
Actual call: create(body={'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/cwl.input.json': {'content': {'y': {'basename': '99999999999999999999999999999998+99', 'location': 'keep:99999999999999999999999999999998+99', 'class': 'Directory'}, 'x': {'basename': u'blorp.txt', 'class': 'File', 'location': u'keep:99999999999999999999999999999994+99/blorp.txt'}, 'z': {'basename': 'anonymous', 'class': 'Directory', 'listing': [{'basename': 'renamed.txt', 'class': 'File', 'location': 'keep:99999999999999999999999999999998+99/file1.txt'}]}}, 'kind': 'json'}}, 'state': 'Committed', 'owner_uuid': None, 'output_path': '/var/spool/cwl', 'name': 'submit_wf.cwl', 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--output-tags=tag0,tag1,tag2', '--enable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/cwl.input.json'], 'cwd': '/var/spool/cwl', 'runtime_constraints': {'API': True, 'vcpus': 1, 'ram': 1073741824}})
======================================================================
FAIL: test_submit_container_runner_ram (tests.test_submit.TestSubmit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 213, in wrapped
return func(self, stubs, *args, **kwargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 488, in test_submit_container_runner_ram
body=expect_container)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 937, in assert_called_with
six.raise_from(AssertionError(_error_message(cause)), cause)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/six.py", line 718, in raise_from
raise value
AssertionError: Expected call: create(body={'name': 'submit_wf.cwl', 'runtime_constraints': {'vcpus': 1, 'API': True, 'ram': 2147483648}, 'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/job/cwl.input.json': {'portable_data_hash': 'd20d7cddd1984f105dd3702c7f125afb+60/cwl.input.json', 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}}, 'state': 'Committed', 'owner_uuid': None, 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--enable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/job/cwl.input.json'], 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'output_path': '/var/spool/cwl', 'cwd': '/var/spool/cwl'})
Actual call: create(body={'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999991+99', 'kind': 'collection'}, '/var/lib/cwl/cwl.input.json': {'content': {'y': {'basename': '99999999999999999999999999999998+99', 'location': 'keep:99999999999999999999999999999998+99', 'class': 'Directory'}, 'x': {'basename': u'blorp.txt', 'class': 'File', 'location': u'keep:99999999999999999999999999999994+99/blorp.txt'}, 'z': {'basename': 'anonymous', 'class': 'Directory', 'listing': [{'basename': 'renamed.txt', 'class': 'File', 'location': 'keep:99999999999999999999999999999998+99/file1.txt'}]}}, 'kind': 'json'}}, 'state': 'Committed', 'owner_uuid': None, 'output_path': '/var/spool/cwl', 'name': 'submit_wf.cwl', 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--enable-reuse', '/var/lib/cwl/workflow/submit_wf.cwl', '/var/lib/cwl/cwl.input.json'], 'cwd': '/var/spool/cwl', 'runtime_constraints': {'API': True, 'vcpus': 1, 'ram': 2147483648}})
======================================================================
FAIL: test_submit_keepref (tests.test_submit.TestSubmit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 213, in wrapped
return func(self, stubs, *args, **kwargs)
File "/home/radhika/arvados/sdk/cwl/tests/test_submit.py", line 552, in test_submit_keepref
body=expect_container)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/mock/mock.py", line 937, in assert_called_with
six.raise_from(AssertionError(_error_message(cause)), cause)
File "/tmp/tmp.328JRcQcJn/VENVDIR/local/lib/python2.7/site-packages/six.py", line 718, in raise_from
raise value
AssertionError: Expected call: create(body={'priority': 1, 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'state': 'Committed', 'owner_uuid': None, 'output_path': '/var/spool/cwl', 'name': 'expect_arvworkflow.cwl#main', 'mounts': {'/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999994+99', 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}, '/var/lib/cwl/job/cwl.input.json': {'portable_data_hash': 'e5454f8ca7d5b181e21ecd45841e3373+58/cwl.input.json', 'kind': 'collection'}, '/var/spool/cwl': {'writable': True, 'kind': 'collection'}}, 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--enable-reuse', '/var/lib/cwl/workflow/expect_arvworkflow.cwl#main', '/var/lib/cwl/job/cwl.input.json'], 'cwd': '/var/spool/cwl', 'runtime_constraints': {'vcpus': 1, 'API': True, 'ram': 1073741824}})
Actual call: create(body={'priority': 1, 'mounts': {'/var/spool/cwl': {'writable': True, 'kind': 'collection'}, 'stdout': {'path': '/var/spool/cwl/cwl.output.json', 'kind': 'file'}, '/var/lib/cwl/workflow': {'portable_data_hash': '99999999999999999999999999999994+99', 'kind': 'collection'}, '/var/lib/cwl/cwl.input.json': {'content': {u'x': u'XxX', 'job_order': None}, 'kind': 'json'}}, 'state': 'Committed', 'owner_uuid': None, 'output_path': '/var/spool/cwl', 'name': 'expect_arvworkflow.cwl#main', 'container_image': 'arvados/jobs:ed9be4cc10c12051bdb1a65210726ab1d688108c', 'command': ['arvados-cwl-runner', '--local', '--api=containers', '--enable-reuse', '/var/lib/cwl/workflow/expect_arvworkflow.cwl#main', '/var/lib/cwl/cwl.input.json'], 'cwd': '/var/spool/cwl', 'runtime_constraints': {'API': True, 'vcpus': 1, 'ram': 1073741824}})
----------------------------------------------------------------------
Ran 38 tests in 6.730s
FAILED (failures=7)
********** !!!!!! sdk/cwl tests FAILED !!!!!! **********
</pre>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=46170
2016-12-07T22:43:19Z
Radhika Chippada
radhika@curoverse.com
<ul></ul><p>Review comments:</p>
<ul>
<li>sdk/cwl tests are passing now</li>
</ul>
<ul>
<li>(If possible) Seems like a name such as data_fetcher might be better for kwargs["fetcher_constructor"] …</li>
</ul>
<ul>
<li>I can now run a workflow at command prompt using <strong>arvados-cwl-runner <a href="https://arvadosapi.com/9tee4-7fd4e-xox74u3j2yrmlk6">9tee4-7fd4e-xox74u3j2yrmlk6</a></strong> by providing the workflow uuid to a-c-r. I am wondering if it would be more user-friendly if it took the uuid as a --workflow arg <br /><pre>
arvados-cwl-runner --workflow <a href="https://arvadosapi.com/9tee4-7fd4e-xox74u3j2yrmlk6">9tee4-7fd4e-xox74u3j2yrmlk6</a>
</pre></li>
</ul>
<ul>
<li>I ran a workflow using arvados-cwl-runner <a href="https://arvadosapi.com/9tee4-7fd4e-xox74u3j2yrmlk6">9tee4-7fd4e-xox74u3j2yrmlk6</a> and it created a CR <a href="https://arvadosapi.com/9tee4-xvhdp-ztpi98zfthww4p7">9tee4-xvhdp-ztpi98zfthww4p7</a> and succeeded. However, the CR does not retain it’s relationship with the workflow. The CR needs the following:<br /><pre>
"properties": {
"template_uuid": "<a href="https://arvadosapi.com/9tee4-7fd4e-xox74u3j2yrmlk6">9tee4-7fd4e-xox74u3j2yrmlk6</a>"
},
</pre></li>
</ul>
<ul>
<li>I ran a workflow using arvados-cwl-runner <a href="https://arvadosapi.com/9tee4-7fd4e-xox74u3j2yrmlk6">9tee4-7fd4e-xox74u3j2yrmlk6</a> and it created a CR <a href="https://arvadosapi.com/9tee4-xvhdp-ztpi98zfthww4p7">9tee4-xvhdp-ztpi98zfthww4p7</a> with name “main”. It should use the workflow name.</li>
</ul>
<ul>
<li>Needs user guide documentation update at "Writing a CWL workflow" with the command to run using a workflow uuid</li>
</ul>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=46280
2016-12-13T17:53:32Z
Radhika Chippada
radhika@curoverse.com
<ul></ul><p>@ <a class="changeset" title="10576: Integrate CWL extensions page, reorganize running and developing docs." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/50a3cf5df8624618cc00308159fde643dd8d6f28">50a3cf5df</a></p>
<ul>
<li>arvcontainer.py
<ul>
<li>Could there be errors during: self.arvrunner.api.workflows().get(uuid=wfuuid).execute() ?</li>
<li>Could there be errors during: yaml.safe_load(wfrecord["definition”]) ?</li>
</ul></li>
</ul>
<ul>
<li>doc: Submitting a workflow to an Arvados cluster
<ul>
<li>It would be nice to say somewhere that ‘--name’ can be used to set a name for the resulting CR and the default name if not provided?</li>
</ul></li>
</ul>
<ul>
<li>doc: Registering a workflow to use in Workbench
<ul>
<li>This is not longer a true statement, correct? The registered workflow can be used to run it at command prompt as well</li>
<li>Can we add ‘--name’ to one of the examples or mention it?</li>
</ul></li>
</ul>
<ul>
<li>doc: Running registered workflows at the command line
<ul>
<li>It appears that “Control a workflow locally” belongs in this page with the “Submit a workflow and wait for results” etc than in the “Writing a CWL workflow”? At least, it should be added in this page also to be complete.</li>
<li>Can we also have one example with no input and mention --name somewhere?<br /><pre> ~/arvados/doc/user/cwl/bwa-mem$ arvados-cwl-runner <a href="https://arvadosapi.com/962eh-7fd4e-660fw86nz0w77mk">962eh-7fd4e-660fw86nz0w77mk</a> </pre></li>
</ul></li>
</ul>
<ul>
<li>doc: Arvados CWL extensions
<ul>
<li>It would be nice to add a bit more explanation on the top explaining what they are used for</li>
<li>string or array of string => string or array of stringS</li>
<li>It is out of scope for this context but arv:RuntimeConstraints => outputDirType with camel case and keep_cache with underscores seems weird; would be nice to stick with one or the other</li>
</ul></li>
</ul>
<ul>
<li>doc: Is user/topics/running-workflow-command-line.html.textile.liquid needed? (It is zero bytes length)</li>
</ul>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=46299
2016-12-13T21:25:43Z
Radhika Chippada
radhika@curoverse.com
<ul></ul><p>Mostly looks good @ <a class="changeset" title="10576: Add ability to run pipeline templates containing CWL from command line. Split out command ..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/3da0fb0eef35b665c9c427fa9acf9b932dd45062">3da0fb0eef</a></p>
<p>The options in the table in user/cwl/cwl-run-options.html.textile.liquid are getting formatted. The following will resolve the issue:</p>
<pre>
-| --basedir BASEDIR| Base directory used to resolve relative references in the input, default to directory of input object file or current directory (if inputs piped/provided on command line).|
-| --version| Print version and exit|
-| --verbose| Default logging|
+| ==--basedir== BASEDIR| Base directory used to resolve relative references in the input, default to directory of input object file or current directory (if inputs piped/provided on command line).|
+| ==--version==| Print version and exit|
+| ==--verbose==| Default logging|
. . .
</pre>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=46301
2016-12-13T21:49:57Z
Peter Amstutz
peter.amstutz@curii.com
<ul></ul><p>Now at <a class="changeset" title="10576: Tweak Docker docs and add pipeline deprecation notice." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/c668e7499c3b2a1561389a06910e33112e7d171d">c668e7499c3b2a1561389a06910e33112e7d171d</a></p>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=46304
2016-12-14T14:41:23Z
Radhika Chippada
radhika@curoverse.com
<ul></ul><p>LGTM</p>
Arvados - Bug #10576: [CWL] Link checking must be able to recognize keep references
https://dev.arvados.org/issues/10576?journal_id=46308
2016-12-14T15:05:05Z
Peter Amstutz
peter.amstutz@curii.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul><p>Applied in changeset arvados|commit:efb3422f119be36e133ffbd14519c5d6c74addc8.</p>