Project

General

Profile

Actions

Feature #9308

closed

[CWL] Crunchrunner supports using writable keep mount for output

Added by Peter Amstutz over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
10/06/2016
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
1.0

Description

Crunchrunner used by crunch v1 for CWL jobs does not support using the writable keep mount as the output directory.

  1. Define a job runtime constraint that requests use of writable keep mount.
  2. Update crunchrunner to use writable keep mount as the output directory if the runtime constraint is set
  3. Update arvados-cwl-runner to set the job runtime constraint for writable keep when there is an appropriate hint in the CWL document (depends on #9307)

Subtasks 1 (0 open1 closed)

Task #10193: Review 9308-crunchrunner-writable-keepResolvedPeter Amstutz10/06/2016

Actions

Related issues

Blocks Arvados - Feature #9307: [CWL] Arvados specific hint to use writable keep mountResolvedPeter Amstutz10/05/2016

Actions
Actions #1

Updated by Peter Amstutz over 6 years ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz over 6 years ago

  • Assigned To set to Peter Amstutz
  • Target version set to 2016-10-12 sprint
Actions #3

Updated by Lucas Di Pentima over 6 years ago

Some observations:

  • File sdk/go/crunchrunner/crunchrunner.go
    • func setupCommand: Is it possible that the file copy return some error? Maybe it’s convenient to check for that and return it if exist.
    • func getKeepTmp: Can be the case of a corrupted json data being read from ‘buf’ making json.Unmarshall return an error?
  • File sdk/cwl/arvados_cwl/arv-cwl-schema.yml
    • There’s a repeated “this” word on the OutputDirType documentation (line 25)
Actions #4

Updated by Peter Amstutz over 6 years ago

  • Status changed from New to Resolved

Applied in changeset arvados|commit:74ec5b86db46257f75ea1eb94c136ce18e65c906.

Actions #5

Updated by Tom Morris over 6 years ago

  • Status changed from Resolved to In Progress

Running bcl2fastq on e51c5 after this was deployed causes the job to deadlock/hang with 4 cores spinning solidly burning sys time with no user time or I/O

Actions #6

Updated by Peter Amstutz over 6 years ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF