Arvados

Diag workflow worked fine with Slurm + Singularity: 9tee4-xvhdp-muiii2mkby0y9n9 and 9tee4-xvhdp-05oml4rackb6tb4.

Now running the big workflow:

$ arvados-cwl-runner --project-uuid 9tee4-j7d0g-uefv7xq2hlwyph5 ./cwl/wgs-processing-wf.cwl ./yml/wgs-processing-wf-chr19.yml
INFO /usr/bin/arvados-cwl-runner 2.3.0.dev20210827152247, arvados-python-client 2.3.0.dev20210827152247, cwltool 3.0.20210319143721
INFO Resolved './cwl/wgs-processing-wf.cwl' to 'file:///home/ward/arvados-tutorial/WGS-processing/cwl/wgs-processing-wf.cwl'
INFO Using cluster 9tee4 (https://workbench2.9tee4.arvadosapi.com/)
INFO Upload local files: "tail.html" "generatereport.py" "head.html" 
INFO Uploaded to 4fce132bc4ae9b4cc844115246a6bd41+175 (9tee4-4zz18-2x31x08s4433o0n)
INFO Using collection cache size 256 MiB
INFO [container WGS processing workflow scattered over samples] submitted container_request 9tee4-xvhdp-5chn41x5n3793gl
INFO Monitor workflow progress at https://workbench2.9tee4.arvadosapi.com/processes/9tee4-xvhdp-5chn41x5n3793gl
...
INFO Final process status is success

Duh, that one reused everything of course. Trying again:

$ arvados-cwl-runner --disable-reuse --project-uuid 9tee4-j7d0g-uefv7xq2hlwyph5 ./cwl/wgs-processing-wf.cwl ./yml/wgs-processing-wf-chr19.yml
INFO /usr/bin/arvados-cwl-runner 2.3.0.dev20210827152247, arvados-python-client 2.3.0.dev20210827152247, cwltool 3.0.20210319143721
INFO Resolved './cwl/wgs-processing-wf.cwl' to 'file:///home/ward/arvados-tutorial/WGS-processing/cwl/wgs-processing-wf.cwl'
INFO Using cluster 9tee4 (https://workbench2.9tee4.arvadosapi.com/)
INFO Upload local files: "generatereport.py" "tail.html" "head.html" 
INFO Using collection 4fce132bc4ae9b4cc844115246a6bd41+175 (9tee4-4zz18-2x31x08s4433o0n)
INFO Using collection cache size 256 MiB
INFO [container WGS processing workflow scattered over samples] submitted container_request 9tee4-xvhdp-o0l519uaanemolg
INFO Monitor workflow progress at https://workbench2.9tee4.arvadosapi.com/processes/9tee4-xvhdp-o0l519uaanemolg
...

INFO Final process status is success

This completed without issue. It ran much faster than the LSF + Singularity job from #17857, there is clearly a performance problem with our LSF setup. Tracking that in https://dev.arvados.org/issues/17857#note-22.