Bug #3123

[Crunch] Pipeline running errors are not reported to user

Added by Peter Amstutz about 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Crunch
Target version:
Start date:
07/07/2014
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
2.0

Description

When arv-run-pipeline-instance submits a component as a job, and the job submission is rejected by API server, the error is not recorded (and thus not reported to the user in Workbench) and the pipeline simply fails with no explanation.

Example case:

I went to run a job which had an invalid commit hash in "minimum_script_version". This fails (reported in bug #3124) with the following error, but the user can't debug why the job failed without access to the crunch-dispatch logs.

2014-06-30_15:02:13.08383 arv-run-pipeline-instance 4399: create job: ["#<ArgumentError: Invalid operand type 'NilClass' fo
r 'in' operator in filters>"] with attributes {:script=>"run-command", :script_parameters=>{:command=>["java", "-Xmx8g", "-
jar", "$(dir $(picard))/CreateSequenceDictionary.jar", "R=$(file $(fasta))", "O=$(basename $(fasta)).dict"], :fasta=>"27793
0109b2dfec9412d489332784605+2009/human_g1k_v37.fasta", :picard=>"88447c464574ad7f79e551070043f9a9+1970"}, :script_version=>
"master", :repository=>"arvados", :nondeterministic=>nil, :output_is_persistent=>false, :runtime_constraints=>{:docker_imag
e=>"arvados/jobs-bwa-samtools"}, :owner_uuid=>"4xphq-tpzed-d6gnynp5uioqnxo", :minimum_script_version=>"96023b1eb656979a4e0e
87f80818f23d23511c1e", :exclude_script_versions=>nil, :no_reuse=>nil}

Subtasks

Task #3202: Review 3123-pipeline-report-job-submit-errorsResolvedPeter Amstutz

Associated revisions

Revision ea1a5e8a
Added by Peter Amstutz about 5 years ago

Merge branch '3123-pipeline-report-job-submit-errors' closes #3123

History

#1 Updated by Peter Amstutz about 5 years ago

  • Description updated (diff)

#2 Updated by Tom Clegg about 5 years ago

  • Story points set to 2.0

#3 Updated by Tom Clegg about 5 years ago

  • Target version set to 2014-08-06 Sprint

#4 Updated by Radhika Chippada about 5 years ago

  • Subject changed from Pipeline running errors are not reported to user to [Crunch] Pipeline running errors are not reported to user
  • Category set to Crunch

#5 Updated by Peter Amstutz about 5 years ago

  • Description updated (diff)

#6 Updated by Peter Amstutz about 5 years ago

  • Description updated (diff)

#7 Updated by Peter Amstutz about 5 years ago

  • Target version changed from 2014-08-06 Sprint to 2014-07-16 Sprint

#8 Updated by Peter Amstutz about 5 years ago

  • Assigned To set to Peter Amstutz

#9 Updated by Ward Vandewege about 5 years ago

  • you seem to have lost a .to_json on line 333 in arv-run-pipeline-instance:
+    body = { :job => job }.merge(jsonified_create_params)

other than that, LGTM. Haven't taken the time to test it, though.

#10 Updated by Anonymous about 5 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:ea1a5e8a4e5370dde8fc10ac3889b81053b11187.

Also available in: Atom PDF