Project

General

Profile

Actions

Bug #5046

closed

Jobs failing to start. Logs show "rm:cannot remove `/tmp/crunch-job/task/compute19.1.keep': Is a directory"

Added by Abram Connelly over 9 years ago. Updated over 9 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Story points:
-

Description

On qr1hi, pipeline instance qr1hi-d1hrv-y4250viwnk8z966 fails quickly after being registered to start. From the logs, here are some of the first errors it looks to have encountered:

2015-01-21_23:40:34 starting: ['srun','--nodelist=compute4,compute16,compute18,compute19,compute28,compute29,compute43,compute48','-D','/tmp','bash','-c','if mount | grep -q $JOB_WORK/; then for i in $JOB_WORK/*keep $CRUNCH_TMP/task/*.keep; do /bin/fusermount -z -u $i; done; fi; sleep 1; rm -rf $JOB_WORK $CRUNCH_INSTALL $CRUNCH_TMP/task $CRUNCH_TMP/src*']
2015-01-21_23:40:35 rm: cannot remove `/tmp/crunch-job/task/compute4.1.keep': Is a directory
2015-01-21_23:40:35 rm: cannot remove `/tmp/crunch-job/task/compute28.1.keep': Is a directory
2015-01-21_23:40:35 rm: cannot remove `/tmp/crunch-job/task/compute43.1.keep': Is a directory
2015-01-21_23:40:35 rm: cannot remove `/tmp/crunch-job/task/compute18.1.keep': Is a directory
2015-01-21_23:40:35 rm: cannot remove `/tmp/crunch-job/task/compute29.1.keep': Is a directory
2015-01-21_23:40:35 rm: cannot remove `/tmp/crunch-job/task/compute48.1.keep': Is a directory
2015-01-21_23:40:35 rm: cannot remove `/tmp/crunch-job/task/compute16.1.keep': Is a directory
2015-01-21_23:40:35 rm: cannot remove `/tmp/crunch-job/task/compute19.1.keep': Is a directory
2015-01-21_23:40:35 srun: error: compute4: task 0: Exited with exit code 1

Log file manifest is 97af21f7764d3eafa33b58bf186879d9+85.


Related issues

Is duplicate of Arvados - Bug #4967: [Crunch] Doesn't cope well with FUSE mounts left hanging around after killing tasks with SIGKILLResolvedBrett Smith01/21/2015Actions
Actions

Also available in: Atom PDF