Arvados-cwl-runner development » History » Revision 2
Revision 1 (Peter Amstutz, 05/10/2018 02:10 PM) → Revision 2/14 (Peter Amstutz, 05/10/2018 02:39 PM)
h1. Arvados-cwl-runner development Arvados-cwl-runner has two modes: submitting a workflow, and actively managing a workflow In submit mode (--submit) it creates a container request which executes arvados-cwl-runner inside a container. The container request uses the docker image "arvados/jobs" and the docker tag is the version of submitting a-c-r instance. The a-c-r instance inside the container actively manages the workflow (--local), and gets information about the container it is running in using containers().current(). When doing development related to workflow execution, you can also use "--local" to run a locally managed workflow using your development code. However, you still need to test that the workflow can be submitted and run in the container. This requires building a development arvados/jobs image with the correct code and version tag. Here's a summary of the development process: # $ virtualenv venv # $ . venv/bin/activate # $ (cd ~/arvados/sdk/cwl && cd ~/arvados/sdk/cw # $ python setup.py develop) develop # _do your development_ # _test locally_ # $ arvados-cwl-runner --local # $ git commit # $ (cd ~/arvados/sdk/cwl && python setup.py develop) cd ~/arvados/build # $ ~/arvados/build/build-dev-docker-jobs-image.sh ./build-dev-docker-jobs-image.sh # _test submit to cluster_ # $ arvados-cwl-runner --submit Important to note: * You need to have ARVADOS_API_HOST an ARVADOS_API_TOKEN in your environment * "setup.py develop" is convenient for editing files in place, but whenever you commit to git remember to re-run setup.py to record the new version number (which is derived from the git commit timestamp). *