Bug #17708

[crunch-run] should be more tolerant of unsupported file types in the output directory

Added by Ward Vandewege 8 months ago. Updated 8 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
05/21/2021
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

Seeing a fatal error in a job, leading to no outputs being saved at all:

error in CaptureOutput: error scanning files to copy to output: Unsupported file type (mode 200000600) in output dir: "/var/spool/cwl/REDACTEDtmp/tmp.fifo.read1" 

That doesn't quite seem reasonable; if there are leftover fifos or named pipes, surely that should not make crunch-run error out. Printing a warning and just skipping the files of unsupported type in the copy job would be a better way to deal with this situation.


Subtasks

Task #17709: review 17708-crunch-run-unsupported-file-typesResolvedWard Vandewege

Associated revisions

Revision 691bb7a4
Added by Ward Vandewege 8 months ago

Merge branch '17708-crunch-run-unsupported-file-types'

closes #17708

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <>

History

#1 Updated by Ward Vandewege 8 months ago

  • Status changed from New to In Progress

#2 Updated by Ward Vandewege 8 months ago

  • Description updated (diff)

#3 Updated by Ward Vandewege 8 months ago

  • Description updated (diff)

#4 Updated by Ward Vandewege 8 months ago

Ready for review at 7c5a4e0eb8529c7bf19a67ac85e52f61dff4ceb0 on branch 17708-crunch-run-unsupported-file-types

Tests are running at https://ci.arvados.org/view/Developer/job/developer-run-tests/2483

#5 Updated by Ward Vandewege 8 months ago

  • Description updated (diff)

#6 Updated by Tom Clegg 8 months ago

#7 Updated by Ward Vandewege 8 months ago

Tom Clegg wrote:

LGTM. I added a test:

17708-crunch-run-unsupported-file-types @ 250b93d70472dacb03400edc62d1d377aec342a2 -- https://ci.arvados.org/view/Developer/job/developer-run-tests/2484/

Thank you for the test, and the quick review! Merged.

#8 Updated by Ward Vandewege 8 months ago

  • Status changed from In Progress to Resolved

#9 Updated by Ward Vandewege 8 months ago

  • Release set to 38

Also available in: Atom PDF