[CWL] Incorporate cwl-runner into arvados/jobs
We want to do this to make it as easy as possible to run CWL workflows in Arvados. Adding it to arvados/jobs makes that simple because we already have infrastructure to keep it up-to-date on clusters.
- Update the arvados/jobs image to be based on Debian jessie.
- Add the cwl-runner package to arvados/jobs.
- Add a cwl-runner crunch_script to the Arvados repository to call arvados_cwl by filling in inputs from the current job's script_parameters.
arvados-cwl-runner --submitoption that submits a job that runs cwl-runner
- User can use
arvados-cwl-runner --submit tool.cwl input.json to submit a CWL job, including uploading any workflow and input dependencies that are not already present on the target Arvados instance. This will create a job to execute the CWL workflow and submits additional jobs that do the actual work so that the workflow can run unattended similarly to existing Arvados pipelines.