Project

General

Profile

Actions

Bug #16139

closed

arvados-cwl-runner 2.0.0 secondaryFiles issue with CWL v1.0

Added by Bryan Cosca over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
02/10/2020
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

bcosca2@lightning-dev1:~/issues/15996/cwl$ arvados-cwl-runner --submit --no-wait --project-uuid=su92l-j7d0g-4i75jw7iivgg3w7 --api=containers --debug gatk-haplotypecaller.cwl testyml/gatk-haplotypecaller.yml 
/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/__init__.py:17: CWLToolDeprecationWarning: 
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020.
Please upgrade your Python as the Python 2.7 version of cwltool won't be
maintained after that date.

  """,  category=CWLToolDeprecationWarning)
INFO /usr/bin/arvados-cwl-runner 2.0.0, arvados-python-client 2.0.0, cwltool 1.0.20190831161204
INFO Resolved 'gatk-haplotypecaller.cwl' to 'file:///data-sdc/bcosca2/issues/15996/cwl/gatk-haplotypecaller.cwl'
DEBUG Creating collection reader for 1e488d8a8de69c5e07b447c8e1ae059b+755
DEBUG Creating collection reader for a3af04432df3d71d22f2fe8be549ba96+5974
INFO Using empty collection d41d8cd98f00b204e9800998ecf8427e+0
INFO Using empty collection d41d8cd98f00b204e9800998ecf8427e+0
INFO Using collection cache size 256 MiB
ERROR Unhandled error:
  string indices must be integers, not str
Traceback (most recent call last):
  File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/main.py", line 833, in main
    logger=_logger)
  File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/executor.py", line 660, in arv_executor
    runnerjob = next(jobiter)
  File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/future/builtins/newnext.py", line 59, in newnext
    return iterator.next()
  File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/arvados_cwl/runner.py", line 649, in job
    self._init_job(job_order, runtimeContext)
  File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/process.py", line 718, in _init_job
    discover_secondaryFiles=getdefault(runtime_context.toplevel, False)))
  File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/builder.py", line 276, in bind_input
    bindings.extend(self.bind_input(f, datum[f["name"]], lead_pos=lead_pos, tail_pos=f["name"], discover_secondaryFiles=discover_secondaryFiles))
  File "/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/builder.py", line 317, in bind_input
    if "$(" in sf["pattern"] or "${" in sf["pattern"]:
TypeError: string indices must be integers, not str

My CWL is good though...

bcosca2@lightning-dev1:~/issues/15996/cwl$ arvados-cwl-runner --validate gatk-haplotypecaller.cwl 
/usr/share/python2.7/dist/python-arvados-cwl-runner/lib/python2.7/site-packages/cwltool/__init__.py:17: CWLToolDeprecationWarning: 
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020.
Please upgrade your Python as the Python 2.7 version of cwltool won't be
maintained after that date.

  """,  category=CWLToolDeprecationWarning)
INFO /usr/bin/arvados-cwl-runner 2.0.0, arvados-python-client 2.0.0, cwltool 1.0.20190831161204
INFO Resolved 'gatk-haplotypecaller.cwl' to 'file:///data-sdc/bcosca2/issues/15996/cwl/gatk-haplotypecaller.cwl'
gatk-haplotypecaller.cwl is valid CWL.


Files

gatk-haplotypecaller.cwl (596 Bytes) gatk-haplotypecaller.cwl Bryan Cosca, 02/07/2020 04:57 PM
gatk-haplotypecaller.yml (169 Bytes) gatk-haplotypecaller.yml Bryan Cosca, 02/07/2020 04:57 PM

Subtasks 1 (0 open1 closed)

Task #16143: Review 16139-cwl-secondaryfiles-fixResolvedPeter Amstutz02/10/2020

Actions
Actions #2

Updated by Bryan Cosca over 2 years ago

I changed the cwlVersion to v1.1 and it submitted successfully.

Actions #3

Updated by Bryan Cosca over 2 years ago

  • Subject changed from arvados-cwl-runner 2.0.0 secondaryFiles issue to arvados-cwl-runner 2.0.0 secondaryFiles issue with CWL v1.0
Actions #4

Updated by Peter Amstutz over 2 years ago

  • Target version set to 2020-02-12 Sprint
  • Assigned To set to Peter Amstutz
  • Status changed from New to In Progress
Actions #6

Updated by Lucas Di Pentima over 2 years ago

This LGTM, thanks!

Actions #7

Updated by Peter Amstutz over 2 years ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved
Actions #8

Updated by Peter Amstutz over 2 years ago

  • Release set to 22
Actions

Also available in: Atom PDF