https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422014-10-01T00:15:45ZArvadosArvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=157222014-10-01T00:15:45ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/15722/diff?detail_id=14491">diff</a>)</li></ul> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=157232014-10-01T01:34:20ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p>See branch 4048-crunch-job-local-script-version</p> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=158812014-10-03T17:52:56ZTom Cleggtom@curii.com
<ul></ul><p>Should verify this is fixed by <a class="issue tracker-2 status-3 priority-4 priority-default closed parent" title="Feature: [Crunch] arv-crunch-job should work in local mode with an arvados-hosted repository (Resolved)" href="https://dev.arvados.org/issues/3775">#3775</a></p> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=158822014-10-03T17:53:16ZTom Cleggtom@curii.com
<ul><li><strong>Target version</strong> deleted (<del><i>Bug Triage</i></del>)</li></ul> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=191472014-12-10T22:17:51ZBrett Smithbrett.smith@curii.com
<ul><li><strong>Subject</strong> changed from <i>arv-run-pipeline-instance --run-job-here cannot run jobs with local script path in script_version.</i> to <i>[Crunch] arv-run-pipeline-instance --run-job-here cannot run jobs with local script path in script_version.</i></li><li><strong>Category</strong> set to <i>Crunch</i></li><li><strong>Assigned To</strong> set to <i>Brett Smith</i></li><li><strong>Target version</strong> set to <i>2015-01-07 sprint</i></li></ul><p>Will verify.</p> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=191512014-12-10T22:38:31ZBrett Smithbrett.smith@curii.com
<ul><li><strong>Subject</strong> changed from <i>[Crunch] arv-run-pipeline-instance --run-job-here cannot run jobs with local script path in script_version.</i> to <i>[Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.</i></li><li><strong>Assigned To</strong> deleted (<del><i>Brett Smith</i></del>)</li><li><strong>Target version</strong> changed from <i>2015-01-07 sprint</i> to <i>Bug Triage</i></li></ul><p>Nope:</p>
<pre>$ arv-run-pipeline-instance --run-jobs-here --template grep_hash_pipeline.json c1_grep::input=c1bad4b39ca5a924e481008009d94e32+210
arv-run-pipeline-instance 15598: create job: ["script_version filter needs repository filter"] with attributes {:job=>{:script=>"grep", :script_parameters=>{:input=>"c1bad4b39ca5a924e481008009d94e32+210", :pattern=>"a"}, :script_version=>"/home/brett/4048/", :repository=>"", :owner_uuid=>"<a href="https://arvadosapi.com/qr1hi-tpzed-5jakibnrp1qpty1">qr1hi-tpzed-5jakibnrp1qpty1</a>", :is_locked_by_uuid=>"<a href="https://arvadosapi.com/qr1hi-tpzed-5jakibnrp1qpty1">qr1hi-tpzed-5jakibnrp1qpty1</a>", :submit_id=>"instance <a href="https://arvadosapi.com/qr1hi-d1hrv-n93q0alr04p80k0">qr1hi-d1hrv-n93q0alr04p80k0</a> rand 2x9u9ba3nc62j", :state=>"Running"}, :find_or_create=>true}
arv-run-pipeline-instance 15598: component c1_grep new job failed
2014-12-10 22:37:51 +0000 -- pipeline_instance <a href="https://arvadosapi.com/qr1hi-d1hrv-n93q0alr04p80k0">qr1hi-d1hrv-n93q0alr04p80k0</a>
c1_grep - -
c2_hash - -
arv-run-pipeline-instance 15598: job creation failed - giving up on this pipeline instance
</pre>
<p>This is happening partly because of the new-ish filters support in the job/create API method, and the way that arv-run-pipeline-instance creates those filters from particular job parameters. I think arv-run-pipeline-instance should not send filters, or try to reuse jobs at all, when it's running local jobs.</p> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=191522014-12-10T22:45:56ZBrett Smithbrett.smith@curii.com
<ul></ul><p>Since it's apparently relevant: my pipeline template has the script directory listed in script_version, and an empty repository field, following <a href="http://doc.arvados.org/user/tutorials/tutorial-firstscript.html" class="external">the example in our documentation</a>.</p> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=191642014-12-11T15:27:19ZBrett Smithbrett.smith@curii.com
<ul></ul><p>If I make a local git repository, and try to run with that as repository, it doesn't work either.</p>
<p>Relevant job definition from the template:</p>
<pre> "c1_grep": {
"script": "grep",
"script_parameters": {
"input": {
"required": true,
"dataclass": "Collection"
},
"pattern": "a"
},
"repository": "/home/brett/4048",
"script_version": "master"
},
</pre>
<p>Running:</p>
<pre>brett@shell.qr1hi:~/4048$ pwd
/home/brett/4048
brett@shell.qr1hi:~/4048$ git rev-list master
5cd5e8502f8213d17965f23c044ef3ee224e1cf2
brett@shell.qr1hi:~/4048$ arv-run-pipeline-instance --run-jobs-here --template grep_hash_pipeline.json c1_grep::input=c1bad4b39ca5a924e481008009d94e32+210
arv-run-pipeline-instance 3334: create job: ["error searching /home/brett/4048 from 'master' to 'master', excluding []"] with attributes {:job=>{:script=>"grep", :script_parameters=>{:input=>"c1bad4b39ca5a924e481008009d94e32+210", :pattern=>"a"}, :script_version=>"master", :repository=>"/home/brett/4048", :owner_uuid=>"<a href="https://arvadosapi.com/qr1hi-tpzed-5jakibnrp1qpty1">qr1hi-tpzed-5jakibnrp1qpty1</a>", :is_locked_by_uuid=>"<a href="https://arvadosapi.com/qr1hi-tpzed-5jakibnrp1qpty1">qr1hi-tpzed-5jakibnrp1qpty1</a>", :submit_id=>"instance <a href="https://arvadosapi.com/qr1hi-d1hrv-hvn98gb0pujulmy">qr1hi-d1hrv-hvn98gb0pujulmy</a> rand 1bjmrbav6cdnh", :state=>"Running"}, :find_or_create=>true}
arv-run-pipeline-instance 3334: component c1_grep new job failed
2014-12-11 15:23:33 +0000 -- pipeline_instance <a href="https://arvadosapi.com/qr1hi-d1hrv-hvn98gb0pujulmy">qr1hi-d1hrv-hvn98gb0pujulmy</a>
c1_grep - -
c2_hash - -
arv-run-pipeline-instance 3334: job creation failed - giving up on this pipeline instance
</pre> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=193572014-12-18T20:16:49ZTom Cleggtom@curii.com
<ul><li><strong>Assigned To</strong> set to <i>Tom Clegg</i></li></ul> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=203092015-01-21T20:33:00ZTom Cleggtom@curii.com
<ul><li><strong>Target version</strong> changed from <i>Bug Triage</i> to <i>2015-01-28 Sprint</i></li></ul> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=206062015-01-28T20:15:13ZWard Vandewegeward@curii.com
<ul><li><strong>Story points</strong> set to <i>0.5</i></li></ul> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=206922015-01-29T18:32:39ZTom Cleggtom@curii.com
<ul><li><strong>Target version</strong> changed from <i>2015-01-28 Sprint</i> to <i>Arvados Future Sprints</i></li></ul> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=530532017-06-27T15:29:28ZTom Cleggtom@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Closed</i></li></ul> Arvados - Bug #4048: [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.https://dev.arvados.org/issues/4048?journal_id=713712019-02-13T18:04:01ZTom Morristfmorris@veritasgenetics.com
<ul><li><strong>Target version</strong> deleted (<del><i>Arvados Future Sprints</i></del>)</li></ul>