Actions
Common Workflow Language integration with Arvados » History » Revision 6
« Previous |
Revision 6/7
(diff)
| Next »
Peter Amstutz, 08/08/2016 03:58 PM
Common Workflow Language integration with Arvados¶
The main integration is arvados-cwl-runner
. This tool is used both as a client to submit a job to run the workflow (arvados-cwl-runner --submit
) and for running the workflow directly (arvados-cwl-runner --local
). When the user starts a CWL workflow from workbench, it functions similar to --submit
mode.
Because submit mode works by creating a "runner" job whose purpose is to manage execution of the workflow, it requires an arvados/jobs
Docker image with an up-to-date arvados-cwl-runner
.
When running a workflow:
- The user submits a workflow by running
arvados-cwl-runner
or through workbench, this creates an "workflow job" - The workflow job is dispatched to a compute node (Compute1)
- The workflow job runs
arvados-cwl-runner
in aarvados/jobs
Docker container with the associated CWL workflow and workflow inputs as job inputs - The runner job is responsible for actually executing the workflow:
- Create a job for each workflow step
- Submit the job to API server (Job A)
- Wait for the job to complete
- Get the output collection for the job (Output A)
- Compute the resulting CWL output object for the workflow step (Result A)
- Iterate over the workflow until all steps have been executed and final workflow output is available
- Return the final workflow result
Updated by Peter Amstutz about 8 years ago · 6 revisions