Project

General

Profile

Actions
Copy link

Feature #9308

closed

[CWL] Crunchrunner supports using writable keep mount for output

Added by Peter Amstutz almost 8 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Peter Amstutz
Category:
-
Target version:
2016-10-12 sprint
Story points:
1.0

Description

Crunchrunner used by crunch v1 for CWL jobs does not support using the writable keep mount as the output directory.

  1. Define a job runtime constraint that requests use of writable keep mount.
  2. Update crunchrunner to use writable keep mount as the output directory if the runtime constraint is set
  3. Update arvados-cwl-runner to set the job runtime constraint for writable keep when there is an appropriate hint in the CWL document (depends on #9307)

Subtasks 1 (0 open — 1 closed)

Task #10193: Review 9308-crunchrunner-writable-keepResolvedPeter Amstutz10/06/2016Actions

Related issues

Blocks Arvados - Feature #9307: [CWL] Arvados specific hint to use writable keep mountResolvedPeter Amstutz10/05/2016Actions
Actions
Copy link
 #1

Updated by Peter Amstutz almost 8 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by Peter Amstutz over 7 years ago

  • Assigned To set to Peter Amstutz
  • Target version set to 2016-10-12 sprint
Actions
Copy link
 #3

Updated by Lucas Di Pentima over 7 years ago

Some observations:

  • File sdk/go/crunchrunner/crunchrunner.go
    • func setupCommand: Is it possible that the file copy return some error? Maybe it’s convenient to check for that and return it if exist.
    • func getKeepTmp: Can be the case of a corrupted json data being read from ‘buf’ making json.Unmarshall return an error?
  • File sdk/cwl/arvados_cwl/arv-cwl-schema.yml
    • There’s a repeated “this” word on the OutputDirType documentation (line 25)
Actions
Copy link
 #4

Updated by Peter Amstutz over 7 years ago

  • Status changed from New to Resolved

Applied in changeset arvados|commit:74ec5b86db46257f75ea1eb94c136ce18e65c906.

Actions
Copy link
 #5

Updated by Tom Morris over 7 years ago

  • Status changed from Resolved to In Progress

Running bcl2fastq on e51c5 after this was deployed causes the job to deadlock/hang with 4 cores spinning solidly burning sys time with no user time or I/O

Actions
Copy link
 #6

Updated by Peter Amstutz over 7 years ago

  • Status changed from In Progress to Resolved
Actions
Copy link

Also available in: Atom PDF