Arvados

This can cover an entire class of analysis tools / utilities with a single crunch script; pipeline/user will specify things like

• Source/binary distribution to download from Keep
• Command line arguments
• How to store/collate output

This should wrap some utility classes for translating script_parameters to command line arguments, along these lines:

• https://github.com/biopython/biopython/blob/master/Bio/Application/__init__.py

These classes should be documented so they can help DRY up other bundled, custom, and third-party crunch scripts.

"mr-file-filter" from warehouse-apps did a simple version of this along these lines:

• provide shell command as a script_parameter
• flag to concatenate all outputs into one file, vs. 1:1 with input