Project

General

Profile

Actions

Bug #11257

closed

[CWL] Fails with $schemas referencing remote resources.

Added by Peter Amstutz about 7 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Story points:
-

Description

Workflows with external http resources listed in $schemas get into trouble with the code to find and upload local dependencies. It should only attempt to upload identifiers with a file:// scheme.

$ arvados-cwl-runner --debug Dockstore.cwl  test.json 
2017-03-15 14:47:25 cwltool INFO: /home/peter/work/scripts/venv/bin/arvados-cwl-runner d363323bfea33aee9182e71fd93387b0c4f944a2 1.0.20170314191734, arvados-python-client 0.1.20170309224426, cwltool 1.0.20170213175853
2017-03-15 14:47:25 cwltool INFO: Resolved 'Dockstore.cwl' to 'file:///home/peter/work/tool-challenge/dockstore-tool-md5sum/Dockstore.cwl'
2017-03-15 14:47:26 cwltool ERROR: Workflow error, try again with --debug for more information:
Input file path 'http://xmlns.com/foaf/spec/20140114.rdf' is invalid
Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/cwltool/main.py", line 728, in main
    **vars(args))
  File "build/bdist.linux-x86_64/egg/arvados_cwl/__init__.py", line 348, in arv_executor
    upload_workflow_deps(self, tool)
  File "build/bdist.linux-x86_64/egg/arvados_cwl/runner.py", line 212, in upload_workflow_deps
    tool.visit(upload_tool_deps)
  File "build/bdist.linux-x86_64/egg/cwltool/process.py", line 599, in visit
    op(self.tool)
  File "build/bdist.linux-x86_64/egg/arvados_cwl/runner.py", line 209, in upload_tool_deps
    include_primary=False)
  File "build/bdist.linux-x86_64/egg/arvados_cwl/runner.py", line 108, in upload_dependencies
    name=name)
  File "build/bdist.linux-x86_64/egg/arvados_cwl/pathmapper.py", line 29, in __init__
    super(ArvPathMapper, self).__init__(referenced_files, input_basedir, None)
  File "build/bdist.linux-x86_64/egg/cwltool/pathmapper.py", line 150, in __init__
    self.setup(dedup(referenced_files), basedir)
  File "build/bdist.linux-x86_64/egg/arvados_cwl/pathmapper.py", line 96, in setup
    self.visit(srcobj, uploadfiles)
  File "build/bdist.linux-x86_64/egg/arvados_cwl/pathmapper.py", line 56, in visit
    raise WorkflowException("Input file path '%s' is invalid" % st)
  File "build/bdist.linux-x86_64/egg/schema_salad/sourceline.py", line 146, in __exit__
    raise self.makeError(unicode(exc_value))
WorkflowException: Input file path 'http://xmlns.com/foaf/spec/20140114.rdf' is invalid

Related issues

Related to Arvados - Bug #17267: Can't run when $schema links are brokenResolvedPeter Amstutz01/18/2021Actions
Actions #1

Updated by Peter Amstutz about 7 years ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz about 7 years ago

Fix pending in #10401 (as part of larger pathmapper refactor)

Actions #3

Updated by Tom Morris over 6 years ago

  • Target version set to Arvados Future Sprints
Actions #4

Updated by Peter Amstutz about 3 years ago

  • Related to Bug #16477: arvados-cwl-runner is missing --skip-schemas added
Actions #5

Updated by Peter Amstutz about 3 years ago

  • Related to Bug #17267: Can't run when $schema links are broken added
Actions #6

Updated by Peter Amstutz about 3 years ago

  • Related to deleted (Bug #16477: arvados-cwl-runner is missing --skip-schemas)
Actions #7

Updated by Peter Amstutz about 3 years ago

  • Status changed from New to Resolved
Actions #8

Updated by Ward Vandewege almost 3 years ago

  • Target version changed from Arvados Future Sprints to 2021-06-23 sprint
Actions

Also available in: Atom PDF