Project

General

Profile

Actions

Bug #4048

closed

[Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.

Added by Peter Amstutz over 9 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
Category:
Crunch
Target version:
-
Story points:
0.5


Related issues

Related to Arvados - Feature #3775: [Crunch] arv-crunch-job should work in local mode with an arvados-hosted repositoryResolvedTom Clegg08/29/2014Actions
Actions #1

Updated by Peter Amstutz over 9 years ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz over 9 years ago

See branch 4048-crunch-job-local-script-version

Actions #3

Updated by Tom Clegg over 9 years ago

Should verify this is fixed by #3775

Actions #4

Updated by Tom Clegg over 9 years ago

  • Target version deleted (Bug Triage)
Actions #5

Updated by Brett Smith over 9 years ago

  • Subject changed from arv-run-pipeline-instance --run-job-here cannot run jobs with local script path in script_version. to [Crunch] arv-run-pipeline-instance --run-job-here cannot run jobs with local script path in script_version.
  • Category set to Crunch
  • Assigned To set to Brett Smith
  • Target version set to 2015-01-07 sprint

Will verify.

Actions #6

Updated by Brett Smith over 9 years ago

  • Subject changed from [Crunch] arv-run-pipeline-instance --run-job-here cannot run jobs with local script path in script_version. to [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local script path in script_version.
  • Assigned To deleted (Brett Smith)
  • Target version changed from 2015-01-07 sprint to Bug Triage

Nope:

$ arv-run-pipeline-instance --run-jobs-here --template grep_hash_pipeline.json  c1_grep::input=c1bad4b39ca5a924e481008009d94e32+210
arv-run-pipeline-instance 15598: create job: ["script_version filter needs repository filter"] with attributes {:job=>{:script=>"grep", :script_parameters=>{:input=>"c1bad4b39ca5a924e481008009d94e32+210", :pattern=>"a"}, :script_version=>"/home/brett/4048/", :repository=>"", :owner_uuid=>"qr1hi-tpzed-5jakibnrp1qpty1", :is_locked_by_uuid=>"qr1hi-tpzed-5jakibnrp1qpty1", :submit_id=>"instance qr1hi-d1hrv-n93q0alr04p80k0 rand 2x9u9ba3nc62j", :state=>"Running"}, :find_or_create=>true}
arv-run-pipeline-instance 15598: component c1_grep new job failed

2014-12-10 22:37:51 +0000 -- pipeline_instance qr1hi-d1hrv-n93q0alr04p80k0
c1_grep -                           -
c2_hash -                           -
arv-run-pipeline-instance 15598: job creation failed - giving up on this pipeline instance

This is happening partly because of the new-ish filters support in the job/create API method, and the way that arv-run-pipeline-instance creates those filters from particular job parameters. I think arv-run-pipeline-instance should not send filters, or try to reuse jobs at all, when it's running local jobs.

Actions #7

Updated by Brett Smith over 9 years ago

Since it's apparently relevant: my pipeline template has the script directory listed in script_version, and an empty repository field, following the example in our documentation.

Actions #8

Updated by Brett Smith over 9 years ago

If I make a local git repository, and try to run with that as repository, it doesn't work either.

Relevant job definition from the template:

        "c1_grep": {
            "script": "grep",
            "script_parameters": {
                "input": {
                    "required": true,
                    "dataclass": "Collection" 
                },
                "pattern": "a" 
            },
            "repository": "/home/brett/4048",
            "script_version": "master" 
        },

Running:

brett@shell.qr1hi:~/4048$ pwd
/home/brett/4048
brett@shell.qr1hi:~/4048$ git rev-list master
5cd5e8502f8213d17965f23c044ef3ee224e1cf2
brett@shell.qr1hi:~/4048$ arv-run-pipeline-instance --run-jobs-here --template grep_hash_pipeline.json  c1_grep::input=c1bad4b39ca5a924e481008009d94e32+210
arv-run-pipeline-instance 3334: create job: ["error searching /home/brett/4048 from 'master' to 'master', excluding []"] with attributes {:job=>{:script=>"grep", :script_parameters=>{:input=>"c1bad4b39ca5a924e481008009d94e32+210", :pattern=>"a"}, :script_version=>"master", :repository=>"/home/brett/4048", :owner_uuid=>"qr1hi-tpzed-5jakibnrp1qpty1", :is_locked_by_uuid=>"qr1hi-tpzed-5jakibnrp1qpty1", :submit_id=>"instance qr1hi-d1hrv-hvn98gb0pujulmy rand 1bjmrbav6cdnh", :state=>"Running"}, :find_or_create=>true}
arv-run-pipeline-instance 3334: component c1_grep new job failed

2014-12-11 15:23:33 +0000 -- pipeline_instance qr1hi-d1hrv-hvn98gb0pujulmy
c1_grep -                           -
c2_hash -                           -
arv-run-pipeline-instance 3334: job creation failed - giving up on this pipeline instance
Actions #9

Updated by Tom Clegg over 9 years ago

  • Assigned To set to Tom Clegg
Actions #10

Updated by Tom Clegg about 9 years ago

  • Target version changed from Bug Triage to 2015-01-28 Sprint
Actions #11

Updated by Ward Vandewege about 9 years ago

  • Story points set to 0.5
Actions #12

Updated by Tom Clegg about 9 years ago

  • Target version changed from 2015-01-28 Sprint to Arvados Future Sprints
Actions #13

Updated by Tom Clegg almost 7 years ago

  • Status changed from New to Closed
Actions #14

Updated by Tom Morris about 5 years ago

  • Target version deleted (Arvados Future Sprints)
Actions

Also available in: Atom PDF