Bug #14650

[arvados-cwl-runner] Inaccessible collection causes long traceback

Added by Tom Morris 5 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Looks like we need better exception handling for this case. It should be a simple "Collection not found" instead of a giant traceback:

Traceback (most recent call last):
  File "/usr/bin/arvados-cwl-runner", line 10, in <module>
    sys.exit(main(sys.argv[1:], sys.stdout, sys.stderr))
  File "/usr/lib/python2.7/dist-packages/arvados_cwl/__init__.py", line 322, in main
    runtimeContext=executor.runtimeContext)
  File "/usr/lib/python2.7/dist-packages/cwltool/main.py", line 776, in main
    secret_store=runtimeContext.secret_store)
  File "/usr/lib/python2.7/dist-packages/cwltool/main.py", line 367, in init_job_order
    visit_class(job_order_object, ("File",), functools.partial(add_sizes, make_fs_access(input_basedir)))
  File "/usr/lib/python2.7/dist-packages/cwltool/utils.py", line 210, in visit_class
    visit_class(rec[d], cls, op)
  File "/usr/lib/python2.7/dist-packages/cwltool/utils.py", line 208, in visit_class
    op(rec)
  File "/usr/lib/python2.7/dist-packages/cwltool/process.py", line 361, in add_sizes
    obj["size"] = fsaccess.size(obj["location"])
  File "/usr/lib/python2.7/dist-packages/arvados_cwl/fsaccess.py", line 150, in size
    collection, rest = self.get_collection(fn)
  File "/usr/lib/python2.7/dist-packages/arvados_cwl/fsaccess.py", line 93, in get_collection
    return (self.collection_cache.get(pdh), urlparse.unquote(sp[1]) if len(sp) == 2 else None)
  File "/usr/lib/python2.7/dist-packages/arvados_cwl/fsaccess.py", line 69, in get
    num_retries=self.num_retries)
  File "/usr/lib/python2.7/dist-packages/arvados/collection.py", line 1823, in __init__
    super(CollectionReader, self).__init__(manifest_locator_or_text, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/arvados/collection.py", line 1309, in __init__
    self._populate()
  File "/usr/lib/python2.7/dist-packages/arvados/collection.py", line 1414, in _populate
    self._populate_from_api_server()
  File "/usr/lib/python2.7/dist-packages/arvados/collection.py", line 1401, in _populate_from_api_server
    num_retries=self.num_retries))
  File "/usr/lib/python2.7/dist-packages/oauth2client/util.py", line 140, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/googleapiclient/http.py", line 840, in execute
    raise HttpError(resp, content, uri=self.uri)
arvados.errors.ApiError: <HttpError 404 when requesting https://e51c5.arvadosapi.com/arvados/v1/collections/25f887d4403065d9080eef682a9b7385%2B70?alt=json returned "Path not found">

Also available in: Atom PDF