Project

General

Profile

Actions

Bug #21039

open

arvados-cwl-runner crashes when running a registered workflow with duplicate enum types

Added by Brett Smith about 1 year ago. Updated about 1 year ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
CWL
Target version:
Story points:
-

Description

Steps to reproduce:

  1. Write a workflow with a separate step file where both use an input with the same enum type.
  2. Register that workflow in Arvados:
    arvados-cwl-runner --create-workflow pathv2.cwl
  3. Submit the registered workflow to Arvados:
    arvados-cwl-runner --submit PATH_V2_CWL_UUID path.yml

arvados-cwl-runner crashes with:

ERROR Workflow execution failed                                                                                                [117/1341]
Traceback (most recent call last): 
  File "schema_salad/avro/schema.py", line 405, in __init__
  File "schema_salad/avro/schema.py", line 595, in make_avsc_object
  File "schema_salad/avro/schema.py", line 373, in __init__
  File "schema_salad/avro/schema.py", line 253, in __init__
  File "schema_salad/avro/schema.py", line 219, in add_name
schema_salad.avro.schema.SchemaParseException: The name 'inpath4d79f6d1-f680-4e05-aa56-c759167a7e80' is already in use.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): 
  File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/executor.py", line 854, in arv_executor                                 
    for runnable in jobiter:
  File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/runner.py", line 896, in job
    self._init_job(job_order, runtimeContext)
  File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/process.py", line 891, in _init_job
    discover_secondaryFiles=getdefault(runtime_context.toplevel, False),
  File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/builder.py", line 335, in bind_input
    discover_secondaryFiles=discover_secondaryFiles,
  File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/builder.py", line 249, in bind_input
    avsc = make_avsc_object(convert_to_dict(t), self.names)
  File "schema_salad/avro/schema.py", line 608, in make_avsc_object
  File "schema_salad/avro/schema.py", line 407, in __init__
schema_salad.avro.schema.SchemaParseException: Items schema ({'symbols': ['_', '_usr', '_usr_local'], 'type': 'enum', 'name': 'inpath4d79f6d1-f680-4e05-aa56-c759167a7e80'}) not a valid Avro schema: The name 'inpath4d79f6d1-f680-4e05-aa56-c759167a7e80' is already in use.. Known names: ['org.w3id.cwl.cwl.File', 'org.w3id.cwl.cwl.File.class.File_class', 'org.w3id.cwl.cwl.Directory', 'org.w3id.cwl.cwl.Directory.class.Directory_class', 'org.w3id.cwl.salad.Any', 'input_record_schema', 'inpath4d79f6d1-f680-4e05-aa56-c759167a7e80', 'outputs_record_schema']).                          
ERROR Workflow error, try again with --debug for more information:
Workflow did not return a result.

Some interesting notes:

  • If you run the workflow without registering it, it works fine.
  • If you run the workflow with --local instead of --submit, it works fine.
  • If you define the enum type in SchemaDefRequirement, it works fine.

Files

buildpathv2.cwl (443 Bytes) buildpathv2.cwl Reproduction tool description Brett Smith, 10/04/2023 05:25 PM
path.yml (32 Bytes) path.yml Reproduction input Brett Smith, 10/04/2023 05:25 PM
pathv2.cwl (419 Bytes) pathv2.cwl Reproduction workflow description Brett Smith, 10/04/2023 05:25 PM
Actions

Also available in: Atom PDF