Bug #3719

[Crunch] run-command crashes: TypeError: coercing to Unicode ... NoneType found

Added by Tom Clegg about 7 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
SDKs
Target version:
Start date:
08/28/2014
Due date:
% Done:

50%

Estimated time:
(Total: 0.00 h)
Story points:
0.5

Subtasks

Task #3721: Improve error reportingResolvedPeter Amstutz

Task #3747: Review 3719-improve-subst-error-reportingResolvedBrett Smith

Associated revisions

Revision 97473ddf
Added by Peter Amstutz about 7 years ago

Merge branch '3719-improve-subst-error-reporting' closes #3719

History

#1 Updated by Abram Connelly about 7 years ago

Pipeline instance qr1hi-d1hrv-uwem8ll6v65vin0 .

Here is the relevant error section taken from the log:

2014-08-27_18:05:23 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr run-command: URL being requested: https://qr1hi.arvadosapi.com/discovery/v1/apis/arvados/v1/rest
2014-08-27_18:05:23 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr run-command: URL being requested: https://qr1hi.arvadosapi.com/arvados/v1/job_tasks/qr1hi-ot0gb-qd7ct4bwsxgyrq9?alt=json
2014-08-27_18:05:23 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr run-command: URL being requested: https://qr1hi.arvadosapi.com/arvados/v1/jobs/qr1hi-8i9sb-0bpb7xclzkdyk8x?alt=json
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr run-command: caught exception
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr Traceback (most recent call last):
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr   File "/tmp/crunch-src/crunch_scripts/run-command", line 158, in <module>
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr     cmd = expand_list(taskp, taskp["command"])
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr   File "/tmp/crunch-src/crunch_scripts/run-command", line 97, in expand_list
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr     return [exp for arg in l for exp in expand_item(p, arg)]
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr   File "/tmp/crunch-src/crunch_scripts/run-command", line 92, in expand_item
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr     return [subst.do_substitution(p, c)]
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr   File "/tmp/crunch-src/crunch_scripts/crunchutil/subst.py", line 63, in do_substitution
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr     v = do_substitution(p, c[m[0]+2 : m[1]])
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr   File "/tmp/crunch-src/crunch_scripts/crunchutil/subst.py", line 73, in do_substitution
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr     c = c[:m[0]] + r + c[m[1]+1:]
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr TypeError: coercing to Unicode: need string or buffer, NoneType found
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr run-command: task parameters was:
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr run-command: {u'command': [u'$(job.srcdir)/crunch_scripts/createChoppedGFFDirList',
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr               u'$(file $(gffDir))'],
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr  u'gffDir': None}
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 stderr srun: error: compute4: task 0: Exited with exit code 1
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 child 21373 on compute4.1 exit 1 signal 0 success=
2014-08-27_18:05:24 qr1hi-8i9sb-0bpb7xclzkdyk8x 21263 0 failure (#1, permanent) after 3 seconds

#2 Updated by Tom Clegg about 7 years ago

  • Target version set to 2014-09-17 sprint

#3 Updated by Tom Clegg about 7 years ago

  • Category set to SDKs
  • Assigned To set to Peter Amstutz

#4 Updated by Peter Amstutz about 7 years ago

  • Status changed from New to In Progress

#5 Updated by Brett Smith about 7 years ago

Reviewing 78317da

  • In SubstitutionError, why override __init__ just to call the super method? You can write pass in the class body if you don't want to override anything.
  • Prefer isinstance(r, basestring) over (isinstance(r, str) or isinstance(r, unicode)).
  • The message "task parameters was:" should probably say "task parameters were:" (both times).

If you make these changes I'm fine with you just merging. Thanks.

#6 Updated by Anonymous about 7 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:97473ddf7cd581a048af7b6585ce3ef75ff502c0.

Also available in: Atom PDF