Arvados

sdk/cwl depends on cwltool==1.0.20160630171631

cwltool==1.0.20160630171631 depends on schema-salad==1.12.20160610104117

schema-salad==1.12.20160610104117 is broken. When installed via pip, it emits an error but succeeds. When installed via fpm-built deb package, the error is fatal.

SyntaxError: ('invalid syntax', ('/usr/lib/python2.7/dist-packages/schema_salad/ref_resolver_BACKUP_28804.py', 222, 2, '<<<<<<< 973dceb3aeef33920968fa257c7b04885a3a716d\n'))

See https://ci.curoverse.com/job/build-packages-debian7/201/consoleText

In other news, there's a new version of ruamel.yaml on pypi, and it's either incompatible with cwltool/schema-salad or just broken.

test_submit_container (tests.test_submit.TestSubmit) ... setup.py 1.0.20160707143733, arvados-python-client 0.1.20160627200331, cwltool 1.0.20160630171631
I'm sorry, I couldn't load this CWL file.  The error was:
Traceback (most recent call last):
  File "/tmp/tmp.yVwDoKykVt/VENVDIR/local/lib/python2.7/site-packages/cwltool/main.py", line 583, in main
    document_loader, workflowobj, uri = fetch_document(args.workflow)
  File "/tmp/tmp.yVwDoKykVt/VENVDIR/local/lib/python2.7/site-packages/cwltool/load_tool.py", line 35, in fetch_document
    workflowobj = document_loader.fetch(fileuri)
  File "/tmp/tmp.yVwDoKykVt/VENVDIR/local/lib/python2.7/site-packages/schema_salad/ref_resolver.py", line 613, in fetch
    result = yaml.load(textIO, Loader=SafeLoader)
  File "/tmp/tmp.yVwDoKykVt/VENVDIR/local/lib/python2.7/site-packages/ruamel/yaml/main.py", line 73, in load
    loader = Loader(stream, version, preserve_quotes=preserve_quotes)
TypeError: __init__() got an unexpected keyword argument 'preserve_quotes'
FAIL

See https://ci.curoverse.com/job/arvados-api-server/3098/consoleText

Looks like this is the YAML parser bug https://bitbucket.org/ruamel/yaml/issues/38
Fixed in ruamel.yaml 0.11.14

Just for the historical record, I think Tom Clegg said via Jabber/XMPP that the YAML parser bug was a red herring and the real issue was something else -- although I don't remember what...