Bug #12418
closed[CWL] Crash with glob on empty collection
Description
2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr Unexpected error from fs_access 2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr Traceback (most recent call last): 2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr File "/usr/lib/python2.7/dist-packages/cwltool/draft2tool.py", line 578, in collect_output 2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr prefix = fs_access.glob(outdir) 2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr File "/usr/lib/python2.7/dist-packages/arvados_cwl/fsaccess.py", line 86, in glob 2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr patternsegments = rest.split("/") 2017-10-08_14:17:52 e51c5-8i9sb-xo9w8fidi304bcv 6764 0 stderr AttributeError: 'NoneType' object has no attribute 'split'
The branch right before this:
if collection and not rest: return [pattern] patternsegments = rest.split("/")
We know rest is None
Seems that bool(collection) is False, this means collection must be None (?) Testing indicates that bool(collection) == True even when collection is empty.
Updated by Tom Morris about 7 years ago
- Status changed from New to In Progress
- Assigned To set to Peter Amstutz
- Target version set to 2017-10-11 Sprint
This is believe to be the underlying cause of the failures in https://projects.veritasgenetics.com/issues/3775
Updated by Bryan Cosca about 7 years ago
This is blocking me here: https://projects.veritasgenetics.com/issues/2968
Also, I added the link to 12323, I posted this error before there.
Updated by Tom Morris about 7 years ago
This is also blocking another four production batches, VGX1991-VGX1994.
Updated by Peter Amstutz about 7 years ago
12418-glob-empty-collection @ 250f1578314d1f4d053d3d9f65a3d5c33d1578af
Use "collection is not None" instead of just "if collection", because the empty collection should evaluate to False.
Strangely, when I tried this in the virtualenv I usually use for development, the empty collection evaluated to True. But in other environments, it evaluates to False (this behavior is more consistent with the documented behavior).
Disassembling the nonzero function indicates that it is failing to check len like it is supposed to in that environment. However it does it correctly in other environments. So this seems to be a Python bug in the particular version in this particular virtualenv (???)
Updated by Lucas Di Pentima about 7 years ago
This LGTM, local test run finished ok. Thanks.
Updated by Peter Amstutz about 7 years ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|commit:0bb435a47e427b12fa2351141a22a1ba1e28a49d.