Common Workflow Language integration with Arvados » History » Version 5

Peter Amstutz, 08/08/2016 02:59 PM

1 1 Peter Amstutz
h1. Common Workflow Language integration with Arvados
2 1 Peter Amstutz
3 3 Peter Amstutz
The main integration is @arvados-cwl-runner@.  This tool is used both as a client to submit a job to run the workflow (@arvados-cwl-runner --submit@) and for running the workflow directly (@arvados-cwl-runner --local@).  When the user starts a CWL workflow from workbench, it functions similar to @--submit@ mode.
4 2 Peter Amstutz
5 2 Peter Amstutz
Because submit mode works by creating a "runner" job whose purpose is to manage execution of the workflow, it requires an @arvados/jobs@ Docker image with an up-to-date @arvados-cwl-runner@.
6 2 Peter Amstutz
7 5 Peter Amstutz
When running a workflow:
8 5 Peter Amstutz
9 5 Peter Amstutz
# The user submits a workflow by running @arvados-cwl-runner@ or through workbench
10 5 Peter Amstutz
# This creates an "workflow job" to run @arvados-cwl-runner@ in the @arvados/jobs@ image with the associated CWL workflow and workflow inputs as job inputs
11 5 Peter Amstutz
# The runner job is responsible for actually executing the workflow:
12 5 Peter Amstutz
## Create a job for each workflow step (Job A)
13 5 Peter Amstutz
## Submit the job to API server
14 5 Peter Amstutz
## Wait for the job to complete
15 5 Peter Amstutz
## Get the output collection for the job 
16 5 Peter Amstutz
## Compute the resulting CWL output object for the workflow step.
17 5 Peter Amstutz
## Iterate over the workflow until all steps have been executed and final workflow output is available
18 5 Peter Amstutz
# Return the final result of the workflow
19 2 Peter Amstutz
20 1 Peter Amstutz
!cwl-arvados-flow-submit.png!