Project

General

Profile

Actions

Bug #12418

closed

[CWL] Crash with glob on empty collection

Added by Peter Amstutz over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Story points:
-

Description

2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr Unexpected error from fs_access
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr Traceback (most recent call last):
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr   File "/usr/lib/python2.7/dist-packages/cwltool/draft2tool.py", line 578, in collect_output
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr     prefix = fs_access.glob(outdir)
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr   File "/usr/lib/python2.7/dist-packages/arvados_cwl/fsaccess.py", line 86, in glob
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr     patternsegments = rest.split("/")
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr AttributeError: 'NoneType' object has no attribute 'split'

The branch right before this:

        if collection and not rest:
            return [pattern]
        patternsegments = rest.split("/")

We know rest is None

Seems that bool(collection) is False, this means collection must be None (?) Testing indicates that bool(collection) == True even when collection is empty.


Subtasks 1 (0 open1 closed)

Task #12421: Review 12418-glob-empty-collectionResolvedPeter Amstutz10/10/2017Actions
Actions #1

Updated by Peter Amstutz over 6 years ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz over 6 years ago

  • Description updated (diff)
Actions #3

Updated by Tom Morris over 6 years ago

  • Status changed from New to In Progress
  • Assigned To set to Peter Amstutz
  • Target version set to 2017-10-11 Sprint

This is believe to be the underlying cause of the failures in https://projects.veritasgenetics.com/issues/3775

Actions #4

Updated by Bryan Cosca over 6 years ago

This is blocking me here: https://projects.veritasgenetics.com/issues/2968

Also, I added the link to 12323, I posted this error before there.

Actions #5

Updated by Tom Morris over 6 years ago

This is also blocking another four production batches, VGX1991-VGX1994.

https://projects.veritasgenetics.com/issues/3773

Actions #6

Updated by Peter Amstutz over 6 years ago

12418-glob-empty-collection @ 250f1578314d1f4d053d3d9f65a3d5c33d1578af

Use "collection is not None" instead of just "if collection", because the empty collection should evaluate to False.

Strangely, when I tried this in the virtualenv I usually use for development, the empty collection evaluated to True. But in other environments, it evaluates to False (this behavior is more consistent with the documented behavior).

Disassembling the nonzero function indicates that it is failing to check len like it is supposed to in that environment. However it does it correctly in other environments. So this seems to be a Python bug in the particular version in this particular virtualenv (???)

Actions #7

Updated by Lucas Di Pentima over 6 years ago

This LGTM, local test run finished ok. Thanks.

Actions #8

Updated by Peter Amstutz over 6 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:0bb435a47e427b12fa2351141a22a1ba1e28a49d.

Actions

Also available in: Atom PDF