Idea #7582
Updated by Peter Amstutz over 8 years ago
CWL jobs are allowed to specify arbitrary Docker containers. In particular, this means depending on our existing run-command script (which depends on the Arvados Python SDK, which itself has long chain of further dependencies) is a non-starter, because we can't assume a friendly environment inside the container.
Implement a simplified run-command (crunch-runner?) in Go that is compiled to a binary with minimum external dependencies that can be loaded into arbitrary containers. This will act as a shim to run the command line generated by CWL and communicate the results to Arvados.
Required features:
- Run arbitrary program in a subprocess
- Set environment variables for subprocess
- Redirect stdin/stdout
- Optionally symlink input files to output directory
- Determine success/failure based on exit code
- Upload contents of output directory to collection
- Communicate task success/failure to API server along with output collection.