Common Workflow Language integration with Arvados » History » Version 7
Peter Amstutz, 08/17/2016 02:33 PM
h1. Common Workflow Language integration with Arvados
The main integration is @arvados-cwl-runner@. This tool is used both as a client to submit a job to run the workflow (@arvados-cwl-runner --submit@) and for running the workflow directly (@arvados-cwl-runner --local@). When the user starts a CWL workflow from workbench, it functions similar to @--submit@ mode.
Because submit mode works by creating a "runner" job whose purpose is to manage execution of the workflow, it requires an @arvados/jobs@ Docker image with an up-to-date @arvados-cwl-runner@.
When running a workflow:
# The user submits a workflow by running @arvados-cwl-runner@ or through workbench, this creates an "workflow orchestrator"
# The workflow orchestrator is dispatched to a compute node (Compute1)
# The workflow orchestrator runs @arvados-cwl-runner@ in a @arvados/jobs@ Docker container with the associated CWL workflow and workflow inputs as job inputs
# The orchestrator job is responsible for actually executing the workflow:
## Create a container request (job) for each workflow step
## Submit the job to API server (Container A)
## Wait for the job to complete
## Get the output collection for the job (Output A)
## Compute the resulting CWL output object for the workflow step (Result A)
## Iterate over the workflow until all steps have been executed and final workflow output is available
# Return the final workflow result