Working plan for changes

    def job(
        self,
        joborder: CWLObjectType,
        output_callback: Optional[OutputCallbackType],
        runtimeContext: RuntimeContext,
    ) -> JobsGeneratorType:
        runtimeContext = runtimeContext.copy()
        runtimeContext.part_of = self.name

        # change custom naming
        for hint in self.step["hints"]:
            if hint["class"] == "http://commonwl.org/cwltool#StepNameHint":
                runtimeContext.name = expression.do_eval(hint["stepname"])
        else:
            runtimeContext.name = shortname(self.id)

        _logger.info("[%s] start", self.name)

        yield from self.step.job(joborder, output_callback, runtimeContext)

My test command line tool, workflow, and input yml are

echo.cwl

cwlVersion: v1.1
class: CommandLineTool
inputs:
  text:
    type: string
    inputBinding: {}
outputs: []
baseCommand: echo

scatter-echo-wf.cwl

$namespaces:
  cwltool: "http://commonwl.org/cwltool#" 
cwlVersion: v1.1
class: Workflow
requirements:
  ScatterFeatureRequirement: {}
  InlineJavascriptRequirement: {}

inputs:
  texts:
    type: string[]

outputs: []

steps:
  echo:
    run: echo.cwl
    scatter: text
    hints:
      cwltool:StepNameHint:
        stepname: $(inputs.text)
    in:
      text: texts
    out: []

scatter-echo-wf.yml

texts: ["a", "b", "c"]

The code I replaced in L72 of workflow_job.py is

runtimeContext.name = expression.do_eval(hint["stepname"], joborder, self.step.requirements, None, None, {})

I'm getting an error when I run "cwltool scatter-echo-wf.cwl scatter-echo-wf.yml".

Traceback (most recent call last):
  File "/home/jiayong/Code/env/cwltool/lib/python3.6/site-packages/cwltool/sandboxjs.py", line 332, in execjs
    return cast(CWLOutputType, json.loads(stdout))
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jiayong/Code/env/cwltool/lib/python3.6/site-packages/cwltool/expression.py", line 414, in do_eval
    else 2,
  File "/home/jiayong/Code/env/cwltool/lib/python3.6/site-packages/cwltool/expression.py", line 308, in interpolate
    js_console=js_console,
  File "/home/jiayong/Code/env/cwltool/lib/python3.6/site-packages/cwltool/expression.py", line 243, in evaluator
    js_console=js_console,
  File "/home/jiayong/Code/env/cwltool/lib/python3.6/site-packages/cwltool/sandboxjs.py", line 338, in execjs
    ) from err
cwltool.sandboxjs.JavascriptException: Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "file:///home/jiayong/Code/work_scripts/cwl/echo/scatter-echo-wf.cwl#echo/text": "a" 
04 };
05 var self = null;
06 var runtime = {
07     "tmpdir": null,
08     "outdir": null
09 };
10 (function(){return ((inputs.text));})()
stdout was: 'undefined'
stderr was: ''

The three key arguments hint["stepname"], joborder, self.step.requirements are as follows

hint["stepname"]: $(inputs.text)
joborder: {'file:///home/jiayong/Code/work_scripts/cwl/echo/scatter-echo-wf.cwl#echo/text': 'a'}
self.step.requirements: [ordereddict([('class', 'InlineJavascriptRequirement')]), ordereddict([('class', 'ScatterFeatureRequirement')])]

Any idea what went wrong there?

Now the problem is that cwltool runs without --enable-ext, but errors out when the flag is turned on.

echo.cwl

cwlVersion: v1.1
class: CommandLineTool
inputs:
  text:
    type: string
    inputBinding: {}
outputs: []
baseCommand: echo

scatter-echo-wf.cwl

$namespaces:
  cwltool: "http://commonwl.org/cwltool#" 
cwlVersion: v1.1
class: Workflow
requirements:
  ScatterFeatureRequirement: {}
  InlineJavascriptRequirement: {}

inputs:
  texts:
    type: string[]

outputs: []

steps:
  echo:
    run: echo.cwl
    scatter: text
    hints:
      cwltool:StepNameHint:
        stepname: $("test_" + inputs.text.split('.')[0])
    in:
      text: texts
    out: []

scatter-echo-wf.yml

texts: ["a.vcf", "b.vcf", "c.vcf"]

Error message:

$ cwltool --enable-ext scatter-echo-wf.cwl scatter-echo-wf.yml 
INFO /home/jiayong/Code/env/cwltool-jiayong/bin/cwltool 3.1.20210511185845
INFO Resolved 'scatter-echo-wf.cwl' to 'file:///home/jiayong/Code/work_scripts/cwl/echo/scatter-echo-wf.cwl'
ERROR Tool definition failed validation:
http://commonwl.org/cwltool:68:3: checking object `http://commonwl.org/cwltool#StepNameHint`
http://commonwl.org/cwltool:74:3:   checking field `fields`
http://commonwl.org/cwltool:81:7:     checking object
                                      `http://commonwl.org/cwltool#StepNameHint/stepname`
                                        Field `type` references unknown identifier `Expression`,
                                        tried http://commonwl.org/cwltool#Expression

I figured out the failed validation is coming from the appended section in extensions-v1.1.yml

- name: StepNameHint
  type: record
  inVocab: false
  extends: cwl:ProcessRequirement
  doc: |
    Provide a hint for naming the runtime workflow step in logs or user interface.
  fields:
    - name: class
      type: string
      doc: "Always 'StepNameHint'" 
      jsonldPredicate:
        "_id": "@type" 
        "_type": "@vocab" 
    - name: stepname
      type: [string, Expression]
      doc: |
        A string or expression returning a string with the preferred name for the step.
        If it is an expression, it is evaluated after the input object has been completely determined.

I'm not sure how I should write the hint differently so the tool definition validation will pass.

Jiayong,

If you merge with the latest cwltool, the reference to cwl:Expression in extensions-v1.1.yml should no longer produce an error.

1. In https://dev.arvados.org/issues/17054#4-Update-WorkflowJobStep-in-workflow_jobpy, you mentioned "checks if the current workflow step has "http://commonwl.org/cwltool#StepNameHint" in "hints" or "requirements"". Right now I'm only checking this under "hints", since it's called "StepNameHint", also the doc field says "provide a hint". Should I expect this to appear under "requirements" as well?

2. I wrote a unit test for custom naming, and it passed. However, some other tests have failed even though I made no changes for them.

=========================== short test summary info ============================
FAILED tests/test_context.py::test_replace_default_stdout_stderr - cwltool.er...
FAILED tests/test_examples.py::test_factory - cwltool.errors.WorkflowExceptio...
FAILED tests/test_load_tool.py::test_check_version - cwltool.errors.WorkflowE...
====== 3 failed, 376 passed, 132 skipped, 2 warnings in 184.91s (0:03:04) ======

Jiayong Li wrote:

1. In https://dev.arvados.org/issues/17054#4-Update-WorkflowJobStep-in-workflow_jobpy, you mentioned "checks if the current workflow step has "http://commonwl.org/cwltool#StepNameHint" in "hints" or "requirements"". Right now I'm only checking this under "hints", since it's called "StepNameHint", also the doc field says "provide a hint". Should I expect this to appear under "requirements" as well?

In general, those extensions are supposed to be available under either hints or requirements, so it is good to accept them in both places for consistency.

You want to be using the method "get_requirement" which searches for a given process requirement (in both "hints" and "requirements") with the correct precedence rules.

2. I wrote a unit test for custom naming, and it passed. However, some other tests have failed even though I made no changes for them.
[...]

Where is your branch so I can review it?

As I said in standup, you should create a pull request for cwltool. In addition to the unit tests there are other code quality tools that are very picky, you will probably need to make further changes to make them happy.

Pull request: https://github.com/common-workflow-language/cwltool/pull/1477