Task #13495
Updated by Peter Amstutz almost 6 years ago
https://dockstore.org/containers/quay.io/pancancer/pcawg_delly_workflow
This is in reference to this email from Denis Yuen trying out the WES endpoint:
> Hi,
>
> Neat, I was able to login and try this out. It works as advertised and I was also able to run a checker workflow that we had handy from the checker workflow tutorial.
>
>
> I ran into some understandable teething issues with larger workflows (in retrospect, they might not be cwl portable, but the issues I'm running into are unrelated) and since you wanted a heads-up, there's some output below. But overall, its nice to see this in action!
>
>
> Commands that worked:
>
> 1) As expected wes-client https://dockstore.org:8443/api/ga4gh/v2/tools/quay.io%2Fbriandoconnor%2Fdockstore-tool-md5sum/versions/1.0.4/plain-CWL/descriptor/%2FDockstore.cwl https://raw.githubusercontent.com/briandoconnor/dockstore-tool-md5sum/master/test.json
>
> 2) Checker workflow worked wes-client https://dockstore.org:8443/api/ga4gh/v2/tools/%23workflow%2Fgithub.com%2Fdockstore-testing%2Fmd5sum-checker%2F_cwl_checker/versions/develop/plain-CWL/descriptor/%2Fchecker-workflow-wrapping-workflow.cwl https://raw.githubusercontent.com/dockstore-testing/md5sum-checker/master/checker-input-cwl.json
>
>
>
>
>
> Commands that were a bit weird (these are versions of delly, these probably aren't CWL portable but I think the errors are too early for that):
> 1) wes-client https://dockstore.org:8443/api/ga4gh/v2/toolsquay.io%2Fpancancer%2Fpcawg_delly_workflow/versions/2.0.1-cwl1.0/plain-CWL/descriptor/%2Fdelly_docker%2FDockstore.cwl https://gist.githubusercontent.com/denis-yuen/de531d704e7521eb8bca669237522b99/raw/4600aa24ff24179f6a8c88e28c4b7995d495be13/modified_dockstore.json
> Traceback (most recent call last):
> File "/usr/local/bin/wes-client", line 11, in <module>
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/wes_client/__init__.py", line 123, in main
> "workflow_type_version": "v1.0"}).result()
> File "/usr/local/lib/python2.7/dist-packages/bravado/http_future.py", line 62, in wrapper
> return func(self, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/bravado/http_future.py", line 121, in result
> self.response_callbacks)
> File "/usr/local/lib/python2.7/dist-packages/bravado/http_future.py", line 153, in unmarshal_response
> raise_on_unexpected(incoming_response)
> File "/usr/local/lib/python2.7/dist-packages/bravado/http_future.py", line 219, in raise_on_unexpected
> raise make_http_exception(response=http_response)
> bravado.exception.HTTPInternalServerError: 500 Internal Server Error
>
> 2) wes-client --no-wait https://dockstore.org:8443/api/ga4gh/v2/tools/quay.io%2Fpancancer%2Fpcawg_delly_workflow/versions/2.0.0-cwl1.0/plain-CWL/descriptor/%2Fdelly_docker%2FDockstore.cwl https://gist.githubusercontent.com/denis-yuen/de531d704e7521eb8bca669237522b99/raw/4600aa24ff24179f6a8c88e28c4b7995d495be13/modified_dockstore.json
> Traceback (most recent call last):
> File "/usr/local/bin/wes-client", line 11, in <module>
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/wes_client/__init__.py", line 123, in main
> "workflow_type_version": "v1.0"}).result()
> File "/usr/local/lib/python2.7/dist-packages/bravado/http_future.py", line 62, in wrapper
> return func(self, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/bravado/http_future.py", line 121, in result
> self.response_callbacks)
> File "/usr/local/lib/python2.7/dist-packages/bravado/http_future.py", line 153, in unmarshal_response
> raise_on_unexpected(incoming_response)
> File "/usr/local/lib/python2.7/dist-packages/bravado/http_future.py", line 219, in raise_on_unexpected
> raise make_http_exception(response=http_response)
> bravado.exception.HTTPGatewayTimeout: 504 Gateway Time-out
>
> I also ran into a few connection pool issues after delly died with the above commands which cleared up after a but, possible load issue?
>
> requests.exceptions.ConnectionError: HTTPSConnectionPool(host='none', port=443): Max retries exceeded with url: /ga4gh/wes/v1/swagger.json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f410479c090>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))
After some conversation on gitter:
> that makes sense, that specific workflow will probably be replaced outright with a new workflow under icgc-argo. At that time, we'll either write them with best practices for network access, /var/spool/cwl, etc. or communicate them to the team that will be doing the work. (So definitely don't put too much work into it)