Story #2342

Provide generic "command line wrapper" crunch script and utility classes

Added by Tom Clegg over 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
07/03/2014
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
1.0
Release relationship:
Auto

Description

This can cover an entire class of analysis tools / utilities with a single crunch script; pipeline/user will specify things like
  • Source/binary distribution to download from Keep
  • Command line arguments
  • How to store/collate output

This should wrap some utility classes for translating script_parameters to command line arguments, along these lines:

These classes should be documented so they can help DRY up other bundled, custom, and third-party crunch scripts.

"mr-file-filter" from warehouse-apps did a simple version of this along these lines:
  • provide shell command as a script_parameter
  • flag to concatenate all outputs into one file, vs. 1:1 with input

Subtasks

Task #3122: Review 2342-run-command-wrapperResolvedTim Pierce

Associated revisions

Revision f6e2ed95 (diff)
Added by Peter Amstutz about 5 years ago

Add 'run-command' generic crunch command wrapper. refs #2342

Revision 2833e72b
Added by Peter Amstutz about 5 years ago

Merge branch '2342-run-command-wrapper' refs #2342

Revision d23fba84
Added by Peter Amstutz about 5 years ago

Merge branch 'resumable-output-upload'. Adds signal handling and error recovery
when uploading job output to run-command wrapper. refs #2342

History

#1 Updated by Tom Clegg over 5 years ago

  • Subject changed from Provide generic "command line wrapper" crunch script, like mr-file-filter from warehouse-apps to Provide generic "command line wrapper" crunch script and utility classes
  • Description updated (diff)

#2 Updated by Tom Clegg over 5 years ago

  • Release changed from 6 to 4

#3 Updated by Tom Clegg about 5 years ago

  • Project changed from Arvados Private to Arvados

#4 Updated by Peter Amstutz about 5 years ago

  • Target version set to 2014-07-16 Sprint

#5 Updated by Peter Amstutz about 5 years ago

  • Assigned To set to Peter Amstutz

#6 Updated by Tim Pierce about 5 years ago

LGTM at 4689df78. thanks.

#7 Updated by Tom Clegg about 5 years ago

  • Status changed from New to In Progress
  • Story points set to 1.0

#8 Updated by Peter Amstutz about 5 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF