Bug #12418

[CWL] Crash with glob on empty collection

Added by Peter Amstutz 2 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
10/10/2017
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-

Description

2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr Unexpected error from fs_access
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr Traceback (most recent call last):
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr   File "/usr/lib/python2.7/dist-packages/cwltool/draft2tool.py", line 578, in collect_output
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr     prefix = fs_access.glob(outdir)
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr   File "/usr/lib/python2.7/dist-packages/arvados_cwl/fsaccess.py", line 86, in glob
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr     patternsegments = rest.split("/")
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr AttributeError: 'NoneType' object has no attribute 'split'

The branch right before this:

        if collection and not rest:
            return [pattern]
        patternsegments = rest.split("/")

We know rest is None

Seems that bool(collection) is False, this means collection must be None (?) Testing indicates that bool(collection) == True even when collection is empty.


Subtasks

Task #12421: Review 12418-glob-empty-collectionResolvedPeter Amstutz

Associated revisions

Revision 0bb435a4
Added by Peter Amstutz 2 months ago

Merge branch '12418-glob-empty-collection' closes #12418

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <>

History

#1 Updated by Peter Amstutz 2 months ago

  • Description updated (diff)

#2 Updated by Peter Amstutz 2 months ago

  • Description updated (diff)

#3 Updated by Tom Morris 2 months ago

  • Status changed from New to In Progress
  • Assigned To set to Peter Amstutz
  • Target version set to 2017-10-11 Sprint

This is believe to be the underlying cause of the failures in https://projects.veritasgenetics.com/issues/3775

#4 Updated by Bryan Cosca 2 months ago

This is blocking me here: https://projects.veritasgenetics.com/issues/2968

Also, I added the link to 12323, I posted this error before there.

#5 Updated by Tom Morris 2 months ago

This is also blocking another four production batches, VGX1991-VGX1994.

https://projects.veritasgenetics.com/issues/3773

#6 Updated by Peter Amstutz 2 months ago

12418-glob-empty-collection @ 250f1578314d1f4d053d3d9f65a3d5c33d1578af

Use "collection is not None" instead of just "if collection", because the empty collection should evaluate to False.

Strangely, when I tried this in the virtualenv I usually use for development, the empty collection evaluated to True. But in other environments, it evaluates to False (this behavior is more consistent with the documented behavior).

Disassembling the nonzero function indicates that it is failing to check len like it is supposed to in that environment. However it does it correctly in other environments. So this seems to be a Python bug in the particular version in this particular virtualenv (???)

#7 Updated by Lucas Di Pentima 2 months ago

This LGTM, local test run finished ok. Thanks.

#8 Updated by Peter Amstutz 2 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:0bb435a47e427b12fa2351141a22a1ba1e28a49d.

Also available in: Atom PDF