Bug #4126

[SDKs] Parameter in pipeline runner is missing

Added by Bryan Cosca about 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Radhika Chippada
Category:
SDKs
Target version:
Start date:
10/09/2014
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
0.5

Description

pipeline_instances/qr1hi-d1hrv-l86cdyufggtu226 here, the log says that first::b - required parameter is missing , even thought you can see in the template that b exists. The pipeline still runs fine and completes with a valid output.

pipeline_instances/qr1hi-d1hrv-d6ix17kcd2fuu0d here, the log says the same thing but the pipeline does not run and fails immediately.

I expected the pipeline to run and complete without this error message because the naming convention worked the first time.


Subtasks

Task #4178: Review 4126-preserve-parameter-hashResolvedPeter Amstutz


Related issues

Precedes Arvados - Bug #4120: [Workbench] [obsolete?] Pipeline state can be failed even though every job is complete.Closed10/10/201410/10/2014

Associated revisions

Revision ff9d78bb
Added by Radhika Chippada about 5 years ago

closes #4126
Merge branch '4126-preserve-parameter-hash'

Revision df54f411 (diff)
Added by Brett Smith about 5 years ago

4126: API server uses fixed a-r-p-i.

Refs #4126.

Revision 153b599f (diff)
Added by Brett Smith about 5 years ago

4126: a-r-p-i dereferences parameter values when creating jobs.

Refs #4126. This is required to run jobs.

Revision 96ca1cad (diff)
Added by Brett Smith about 5 years ago

4126: API server uses fixed a-r-p-i (again).

Refs #4126.

Revision 61790a81 (diff)
Added by Brett Smith about 5 years ago

4126: API server uses fixed a-r-p-i.

Refs #4126.

Revision d7f9b8fd (diff)
Added by Brett Smith about 5 years ago

4126: a-r-p-i dereferences parameter values when creating jobs.

Refs #4126. This is required to run jobs.

Revision 286f4183 (diff)
Added by Brett Smith about 5 years ago

4126: API server uses fixed a-r-p-i (again).

Refs #4126.

History

#1 Updated by Ward Vandewege about 5 years ago

There are other pipeline instances where it complains with that error in the logs, e.g. qr1hi-d1hrv-wji6s57f4k3sqix. Weirdly, the pipeline instance status is 'failed' though the 2 jobs in the pipeline are listed as 'success'.

#2 Updated by Ward Vandewege about 5 years ago

  • Target version set to Bug Triage

#3 Updated by Radhika Chippada about 5 years ago

  • Subject changed from Parameter in pipeline runner is missing to [SDKs] Parameter in pipeline runner is missing
  • Category set to SDKs

I think it is arv-run-pipeline-instance issue and so I set the category as SDKs.

#4 Updated by Radhika Chippada about 5 years ago

I could be wrong, but it appears that "someone" is using the pipeline instance as pipeline template? If you look at the log for pipeline_instances/qr1hi-d1hrv-d6ix17kcd2fuu0d, it says "2014-10-07 18:41:41 +0000 -- pipeline_template qr1hi-d1hrv-d6ix17kcd2fuu0d Errors . . . " where the uuid printed is the instance uuid, not the template uuid.

I pointed my dev env to qr1hi and inserted some debug statements and the value for "b" is definitely there at least during the creation of the instance (I used --submit as well as --run-pipeline-here) and in both cases the value of "b" is the one provided as "value" in the pipeline template.

However, it appears that it is not found when it is actually run (later when it is scheduled in case of --submit for example) and the uuid shown in log tab looks suspicious (instance's instead of template's).

#5 Updated by Radhika Chippada about 5 years ago

Some more observations about this issue.

- The pipeline instance being created from the template seems correct --- :b=>{:value=>{:group=>"e", :regex=>"(.*)_[12]\\.fastq\\.gz$"}}

#<PipelineInstance:0x00000004f2c160 @attributes_to_update={}, @pi={:href=>"/pipeline_instances/qr1hi-d1hrv-jsx0xg30okarj68", :kind=>"arvados#pipelineInstance", :etag=>"115l0h2qo261d4djsjgpivoti", :uuid=>"qr1hi-d1hrv-jsx0xg30okarj68", :owner_uuid=>"qr1hi-tpzed-ktpvhqu89qoib9f", :created_at=>"2014-10-08T02:11:06Z", :modified_by_client_uuid=>"qr1hi-ozdt8-obw7foaks3qjyej", :modified_by_user_uuid=>"qr1hi-tpzed-ktpvhqu89qoib9f", :modified_at=>"2014-10-08T02:11:07Z", :pipeline_template_uuid=>"qr1hi-p5p6p-ujw7clygo3vtorv", :pipeline_template=>{:href=>"/pipeline_templates/qr1hi-p5p6p-ujw7clygo3vtorv", :kind=>"arvados#pipelineTemplate", :etag=>"bf90157j7q69dlqqwfczwzuvn", :uuid=>"qr1hi-p5p6p-ujw7clygo3vtorv", :owner_uuid=>"qr1hi-tpzed-rv6f0l8lbvdzkog", :created_at=>"2014-09-10T17:49:08Z", :modified_by_client_uuid=>"qr1hi-ozdt8-obw7foaks3qjyej", :modified_by_user_uuid=>"qr1hi-tpzed-rv6f0l8lbvdzkog", :modified_at=>"2014-10-07T18:41:10Z", :name=>"mxn test", :components=>{:first=>{:script=>"run-command", :script_version=>"4042-run-command-MxN", :repository=>"bcosc", :script_parameters=>{:command=>["bwa", "mem", "-t", "$(node.cores)", "-R", "@RG\\tID:$(basename $(eachread))\\tPL:illumina\\tSM:$(sample_id)", "$(glob $(dir $(reference))/*.fasta)", {:foreach=>"b", :command=>["$(glob $(b)/*_1.fastq*)", "$(glob $(b)/*_2.fastq*)"]}], :reference=>{:required=>true, :dataclass=>"Collection", :default=>"3514b8e5da0e8d109946bc809b20a78a+5698", :link_name=>"human_g1k_v37 reference data", :title=>"Reference genome (FASTA)"}, :sample_id=>{:required=>true, :default=>"my_sample", :dataclass=>"text", :title=>"Sample identifier"}, :reads=>{:output_of=>"split"}, :"task.foreach"=>["b"], :"task.stdout"=>"$(basename $(basename $(glob $(eachread)/*_1.fastq*))).sam", :b=>{:value=>{:group=>"e", :regex=>"(.*)_[12]\\.fastq\\.gz$"}}, :d=>{:required=>true, :default=>"d0136bc494c21f79fc1b6a390561e6cb+2778", :title=>"input collection", :dataclass=>"Collection"}, :e=>"$(dir $(d))"}, :runtime_constraints=>{:docker_image=>"arvados/jobs-java-bwa-samtools"}, :output_name=>false}}, :description=>nil}, :name=>nil, :components=>{:first=>{:script=>"run-command", :script_version=>"4042-run-command-MxN", :repository=>"bcosc", :script_parameters=>{:command=>["bwa", "mem", "-t", "$(node.cores)", "-R", "@RG\\tID:$(basename $(eachread))\\tPL:illumina\\tSM:$(sample_id)", "$(glob $(dir $(reference))/*.fasta)", {:foreach=>"b", :command=>["$(glob $(b)/*_1.fastq*)", "$(glob $(b)/*_2.fastq*)"]}], :reference=>"3514b8e5da0e8d109946bc809b20a78a+5698", :sample_id=>"my_sample", :reads=>{:output_of=>"split"}, :"task.foreach"=>["b"], :"task.stdout"=>"$(basename $(basename $(glob $(eachread)/*_1.fastq*))).sam", :b=>nil, :d=>"d0136bc494c21f79fc1b6a390561e6cb+2778", :e=>"$(dir $(d))"}, :runtime_constraints=>{:docker_image=>"arvados/jobs-java-bwa-samtools"}, :output_name=>false}}, :dependencies=>["3514b8e5da0e8d109946bc809b20a78a+5698", "d0136bc494c21f79fc1b6a390561e6cb+2778"], :properties=>{:run_options=>{:enable_job_reuse=>true}}, :state=>"RunningOnClient", :components_summary=>{:todo=>1, :done=>0, :failed=>0}, :description=>"Created at 2014-10-07 22:11:06 -0400 using the pipeline template *mxn test*", :started_at=>"2014-10-08T02:11:07Z", :finished_at=>nil}>

- However, when the instance is fetched to run, the apply_parameters method in a-r-p-i is not able to see the value of "b".

- We have the following starting at line 437

        parameter = { :value => parameter } unless parameter.is_a? Hash
        value =
          (params["#{componentname}::#{parametername}"] ||
           parameter[:value] ||
           (parameter[:output_of].nil? &&
            (params[parametername.to_s] ||
             parameter[:default])) ||
           nil)

- I think we need the following (between lines 444 and 445) to account for the case where the "value" is a Hash:

value = component if value.nil? && !component.nil?

#6 Updated by Radhika Chippada about 5 years ago

While on the subject, I think we should fix the log message as well. Currently, the log message looks as:

2014-10-07 18:41:41 +0000 -- pipeline_template qr1hi-d1hrv-d6ix17kcd2fuu0d
Errors:
first::b - required parameter is missing

- We should fix it to say

 2014-10-07 18:41:41 +0000 -- pipeline qr1hi-d1hrv-d6ix17kcd2fuu0d 
or use the word template or instance correctly depending on the object class.

#7 Updated by Tom Clegg about 5 years ago

I think this is (partly) a pipeline template bug.

There are two ways to specify a parameter for a pipeline component.

  1. "paramname":"xyzzy" -- this is a shortcut only suitable for undocumented non-user-editable parameters.
  2. "paramname":{"value":"xyzzy","required":true,"title":"Magic spell to use"} -- this is the general form.

The first form cannot be used when the parameter value is a hash, because it cannot be distinguished from the second form.

Radhika Chippada wrote:

While on the subject, I think we should fix the log message as well. Currently, the log message looks as:

[...]

- We should fix it to say [...] or use the word template or instance correctly depending on the object class.

Yes, there are several instances in the code where we say "template" even though @template is sometimes just a copy of @instance. We should detect this, and say "template" or "instance" depending on which it is.

#8 Updated by Radhika Chippada about 5 years ago

  • Status changed from New to In Progress
  • Assigned To set to Radhika Chippada
  • Target version changed from Bug Triage to 2014-10-08 sprint

#9 Updated by Ward Vandewege about 5 years ago

  • Story points set to 0.5

#10 Updated by Ward Vandewege about 5 years ago

  • Target version changed from 2014-10-08 sprint to 2014-10-29 sprint

#11 Updated by Radhika Chippada about 5 years ago

I was able to create an instance from the template and execute a run-jobs-here command without getting "first::b - required parameter is missing" error by updating the template as listed below.

Please review these changes. Here is what changed:

- Tom updated arv-run-pipeline-instace:

        component[:script_parameters][parametername] =
          parameter.dup.merge(value: value)

- I updated the template as follows:

     "b":{
      "value":{
       "group":"e",
       "regex":"(.*)_[12]\\.fastq\\.gz$" 
      }

#12 Updated by Radhika Chippada about 5 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:ff9d78bbeb09ca917f0814c6055c47db5a4e7e4a.

Also available in: Atom PDF