Feature #7328

Updated by Brett Smith almost 5 years ago

h2. Original report

for example, a simple "ls" in the directory that they are currently in or looking into would help. Working with collections is a new concept, so users can easily run into file not found issues. Also, if they are piping commands and lose track half way, it would be great to get more insight into what is being created/deleted/etc.

h2. Implementation

Overview: Add an excepthook function to the Python SDK that saves the contents of the $TASK_WORK directory as the task's output, and mark it failed. The SDK automatically installs this function as sys.excepthook when it's running under a job that has a debugging runtime constraint set.

* Add a new function to the arvados.crunch Python SDK module.
** It will be installed as "sys.excepthook":https://docs.python.org/2/library/sys.html#sys.excepthook and must take the same signature: @(exc_type, exc_value, traceback)@
** It creates a new collection from the contents of the $TASK_WORK directory, and updates arvados.current_task() so that the task's output is the manifest of that collection, and success is false.
*** run-command already has most of the code to do this.
** No matter what happens in *** TBD: Should it skip this step if @isinstance(exc_value, arvados.errors.KeepRequestError)@? If the above step(s), task failed with a Keep error, creating the collection is unlikely to work.
** The
last thing it does is always call "sys.__excepthook__":https://docs.python.org/2/library/sys.html#sys.__excepthook__ to get the usual exception handling behavior. Ensure this happens using a try/finally block.
* Add this code or the functional equivalent to @__init__.py@:
<pre><code class="python">if os.environ.get('TASK_UUID'):
_want_debughook = current_job()['runtime_constraints']['debug_exceptions']
except (errors.ApiError, KeyError):
# The job doesn't exist or doesn't define the constraint.
if _want_debughook:
sys.excepthook = arvados.crunch.your_new_function
del _want_debughook
* Add documentation for the new runtime constraint to the Job schema page.