Common Workflow Language integration with Arvados » History » Revision 6

Revision 5 (Peter Amstutz, 08/08/2016 02:59 PM) → Revision 6/7 (Peter Amstutz, 08/08/2016 03:58 PM)

h1. Common Workflow Language integration with Arvados 

 The main integration is @arvados-cwl-runner@.    This tool is used both as a client to submit a job to run the workflow (@arvados-cwl-runner --submit@) and for running the workflow directly (@arvados-cwl-runner --local@).    When the user starts a CWL workflow from workbench, it functions similar to @--submit@ mode. 

 Because submit mode works by creating a "runner" job whose purpose is to manage execution of the workflow, it requires an @arvados/jobs@ Docker image with an up-to-date @arvados-cwl-runner@. 

 When running a workflow: 

 # The user submits a workflow by running @arvados-cwl-runner@ or through workbench, this workbench 
 # This creates an "workflow job" 
 # The workflow job is dispatched to a compute node (Compute1) 
 # The workflow job runs run @arvados-cwl-runner@ in a the @arvados/jobs@ Docker container image with the associated CWL workflow and workflow inputs as job inputs 
 # The runner job is responsible for actually executing the workflow: 
 ## Create a job for each workflow step  
 (Job A) 
 ## Submit the job to API server (Job A) 
 ## Wait for the job to complete 
 ## Get the output collection for the job (Output A) 
 ## Compute the resulting CWL output object for the workflow step (Result A) step. 
 ## Iterate over the workflow until all steps have been executed and final workflow output is available 
 # Return the final result of the workflow result