Arvados-cwl-runner development » History » Version 2
Peter Amstutz, 05/10/2018 02:39 PM
1 | 1 | Peter Amstutz | h1. Arvados-cwl-runner development |
---|---|---|---|
2 | |||
3 | 2 | Peter Amstutz | Arvados-cwl-runner has two modes: submitting a workflow, and actively managing a workflow |
4 | |||
5 | In submit mode (--submit) it creates a container request which executes arvados-cwl-runner inside a container. The container request uses the docker image "arvados/jobs" and the docker tag is the version of submitting a-c-r instance. The a-c-r instance inside the container actively manages the workflow (--local), and gets information about the container it is running in using containers().current(). |
||
6 | |||
7 | When doing development related to workflow execution, you can also use "--local" to run a locally managed workflow using your development code. However, you still need to test that the workflow can be submitted and run in the container. This requires building a development arvados/jobs image with the correct code and version tag. |
||
8 | |||
9 | Here's a summary of the development process: |
||
10 | |||
11 | 1 | Peter Amstutz | # $ virtualenv venv |
12 | # $ . venv/bin/activate |
||
13 | 2 | Peter Amstutz | # $ (cd ~/arvados/sdk/cwl && python setup.py develop) |
14 | 1 | Peter Amstutz | # _do your development_ |
15 | 2 | Peter Amstutz | # _test locally_ |
16 | # $ arvados-cwl-runner --local |
||
17 | 1 | Peter Amstutz | # $ git commit |
18 | 2 | Peter Amstutz | # $ (cd ~/arvados/sdk/cwl && python setup.py develop) |
19 | # $ ~/arvados/build/build-dev-docker-jobs-image.sh |
||
20 | # _test submit to cluster_ |
||
21 | # $ arvados-cwl-runner --submit |
||
22 | |||
23 | Important to note: |
||
24 | |||
25 | * You need to have ARVADOS_API_HOST an ARVADOS_API_TOKEN in your environment |
||
26 | * "setup.py develop" is convenient for editing files in place, but whenever you commit to git remember to re-run setup.py to record the new version number (which is derived from the git commit timestamp). |
||
27 | * |