Arvados

Stack trace from job qr1hi-8i9sb-1fmcoid6lfess9g:

1/5/2015 5:40:24 PM    compute20    1    task-print    0    Traceback (most recent call last):
1/5/2015 5:40:24 PM    compute20    1    task-print    0    File "/tmp/crunch-src/crunch_scripts/bwa-index", line 11, in <module>
1/5/2015 5:40:24 PM    compute20    1    task-print    0    ref_dir = arvados.util.collection_extract(collection = this_job['script_parameters']['input'], path = 'reference', decompress = False)
1/5/2015 5:40:24 PM    compute20    1    task-print    0    File "/usr/local/lib/python2.7/dist-packages/arvados_python_client-0.1-py2.7.egg/arvados/util.py", line 201, in collection_extract
1/5/2015 5:40:24 PM    compute20    1    task-print    0    lockfile = open(path + '.lock', 'w')
1/5/2015 5:40:24 PM    compute20    1    task-print    0    IOError: [Errno 2] No such file or directory: '/tmp/crunch-job/work/reference.lock'

Possible diagnosis: Crunch jobs are run with JOB_WORK and TASK_WORK environment variables set to temporary working directories. These are made available to jobs as arvados.current_job().tmpdir and arvados.current_task().tmpdir respectively. However, when these jobs run in Docker, the JOB_WORK directory is not guaranteed to exist.