Running Common Workflow Language (CWL) workflows on Arvados¶

Common Workflow Language is an emerging standard for describing portable, cross-platform workflows. The Arvados project has been involved in this effort from the very beginning, and is pleased to offer preliminary support for running CWL in Arvados.

The Arvados CWL runner is located at "arvados/sdk/cwl" in the Arvados source tree. The Python package is called "arvados-cwl-runner". You can also install from PyPi:

$ pip install arvados-cwl-runner

If you are using a shell account on a Curoverse-hosted cluster, your environment is already set up. Otherwise, first set up your environment (see also Arvswitch to help manage your Arvados credentials.)

If a pipeline inputs refer to local files, they will be uploaded before running the pipeline. You can also reference files already in Keep by prefixing "keep:" for example "keep:d341a6f1db391a780d694e240e95e475+3805/lobstr_v3.0.2_hg19_ref/lobSTR_ref.fasta".

See the lobSTR pipeline for an example.