Feature #17054

Custom naming for scatter steps

Added by Peter Amstutz 6 months ago. Updated 4 days ago.

Status:
New
Priority:
Normal
Assigned To:
Category:
CWL
Start date:
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Story points:
-

Description

Add an extension to cwltool that allows the user to provide an expression that will determine the runtime name of a workflow step or scatter step. When a new cwltool is released, update the dependency arvados-cwl-runner.

Suggested approach

1. Add a new process requirement to cwltool/extensions-v1.1.yml

- name: StepNameHint
  type: record
  inVocab: false
  extends: cwl:ProcessRequirement
  doc: |
    Provide a hint for naming the runtime workflow step in logs or user interface.
  fields:
    - name: class
      type: string
      doc: "Always 'StepNameHint'" 
      jsonldPredicate:
        "_id": "@type" 
        "_type": "@vocab" 
    - name: stepname:
      type: [string, Expression]
      doc: |
        A string or expression returning a string with the preferred name for the step.  
        If it is an expression, it is evaluated after the input object has been completely determined.

2. update supportedProcessRequirements

Add "http://commonwl.org/cwltool#StepNameHint" to process.py supportedProcessRequirements

3. Update setup_schema() in main.py

use_custom_schema("v1.2", "http://commonwl.org/cwltool", ext11)

you should also add this to the "else" branch:
use_standard_schema("v1.2")

4. Update WorkflowJobStep in workflow_job.py

Add code to the job() method that

  1. checks if the current workflow step has "http://commonwl.org/cwltool#StepNameHint" in "hints" or "requirements"
  2. If so, gets the value of "stepname"
  3. Then does self.name = expression.do_eval(stepname)

5. Add tests

Write a workflow that uses the new hint to with an expression that uses something from the input to set the name of the workflow step.

Write a test case that calls cwltool --enable-ext and checks that the log output uses the custom name.


Subtasks

Task #17456: ReviewNewLucas Di Pentima


Related issues

Related to Arvados Epics - Story #16011: Easier CWLIn Progress07/01/202007/01/2021

Related to Arvados - Feature #16462: Expand arvados-controller to expose forecast featuresNew

History

#1 Updated by Peter Amstutz 6 months ago

#2 Updated by Nico C├ęsar 6 months ago

  • Related to Feature #16462: Expand arvados-controller to expose forecast features added

#3 Updated by Peter Amstutz about 1 month ago

  • Target version set to 2021-03-31 sprint
  • Assigned To set to Jiayong Li

#4 Updated by Peter Amstutz about 1 month ago

  • Description updated (diff)

#5 Updated by Peter Amstutz 23 days ago

  • Description updated (diff)

#6 Updated by Peter Amstutz 18 days ago

  • Target version changed from 2021-03-31 sprint to 2021-04-14 sprint

#7 Updated by Peter Amstutz 4 days ago

  • Target version changed from 2021-04-14 sprint to 2021-04-28 bughunt sprint

#8 Updated by Peter Amstutz 4 days ago

  • Target version deleted (2021-04-28 bughunt sprint)

#9 Updated by Peter Amstutz 4 days ago

  • Target version set to 2021-04-28 bughunt sprint

Also available in: Atom PDF