Project

General

Profile

Actions

Bug #13140

closed

[CWL] Update cwltool for latest cwl conformance tests

Added by Peter Amstutz over 6 years ago. Updated about 6 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Story points:
-
Release relationship:
Auto

Description

Some conformance tests are failing. Update arvados-cwl-runner to pass.


Subtasks 2 (0 open2 closed)

Task #13154: Update cwltool & ensure passing testsResolvedPeter Amstutz03/02/2018Actions
Task #13161: Review 13140-cwl-conformanceResolvedPeter Amstutz03/02/2018Actions
Actions #1

Updated by Peter Amstutz over 6 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Peter Amstutz over 6 years ago

  • Subject changed from [CWL] Update cwltool to [CWL] Update cwltool for latest cwl conformance tests
Actions #3

Updated by Peter Amstutz over 6 years ago

  • Description updated (diff)
Actions #4

Updated by Peter Amstutz over 6 years ago

  • Status changed from In Progress to New
  • Assigned To set to Peter Amstutz
Actions #5

Updated by Lucas Di Pentima over 6 years ago

Just to keep the record of what we already chatted:

Tried to run the conformance tests on arvbox and got errors similar to:

Test [13/127] 
Test failed: /home/lucas/venv/bin/arvados-cwl-runner --api=containers --compute-checksum --disable-reuse --outdir=/tmp/tmp0wBPwn --quiet v1.0/cat4-tool.cwl v1.0/cat-job.json
Test command execution in Docker with stdin and stdout redirection
Returned non-zero
2018-03-02 16:04:26 cwltool ERROR: Workflow error, try again with --debug for more information:
v1.0/cat4-tool.cwl:7:5: 'module' object has no attribute 'get_image'

There are also some tests failing, don't know if they're related: https://ci.curoverse.com/job/developer-run-tests/626/

Actions #6

Updated by Peter Amstutz over 6 years ago

Thanks.

Updated with fix 13140-cwl-conformance @ cde5fc115a5d543eb6522fdad6e651dfbe857656

Actions #7

Updated by Lucas Di Pentima over 6 years ago

I'm having problems when trying to run the conformance tests on arvbox:

(venv) lucas@veritas:~/common-workflow-language$ ./run_test.sh RUNNER=arvados-cwl-runner EXTRA="--api=containers --compute-checksum --disable-reuse" 
--- Running conformance test v1.0 on /home/lucas/venv/bin/arvados-cwl-runner ---
/home/lucas/venv/bin/arvados-cwl-runner 1.0.20180302211217, arvados-python-client 0.1.20180223161544, cwltool 1.0.20180225105849
Test [1/127] 
Test timed out: /home/lucas/venv/bin/arvados-cwl-runner --api=containers --compute-checksum --disable-reuse --outdir=/tmp/tmp_fJ2Wz --quiet v1.0/bwa-mem-tool.cwl v1.0/bwa-mem-job.json
General test of command line generation
Test [2/127] 

Maybe it's a local issue, I tried resetting the arvbox instance from scratch, re-building the image, but it's not working for me. On workbench the jobs appear as Queued and are kept that way after the timeout.

Actions #8

Updated by Lucas Di Pentima over 6 years ago

As suggested by Peter I asked for the crunch-dispatch-local logs and I'm seeing this error:

[...]
2018-03-05_19:01:10.61051 # git.curoverse.com/arvados.git/services/crunch-run
2018-03-05_19:01:10.61054 src/git.curoverse.com/arvados.git/services/crunch-run/crunchrun.go:1777: cannot use docker (type *client.Client) as type ThinDockerClient in argument to NewContaine
rRunner:
2018-03-05_19:01:10.61055       *client.Client does not implement ThinDockerClient (wrong type for ContainerAttach method)
2018-03-05_19:01:10.61055               have ContainerAttach("github.com/docker/docker/vendor/golang.org/x/net/context".Context, string, types.ContainerAttachOptions) (types.HijackedResponse
, error)
2018-03-05_19:01:10.61055               want ContainerAttach("golang.org/x/net/context".Context, string, types.ContainerAttachOptions) (types.HijackedResponse, error)
[...]
Actions #9

Updated by Peter Amstutz over 6 years ago

Lucas Di Pentima wrote:

As suggested by Peter I asked for the crunch-dispatch-local logs and I'm seeing this error:

[...]

arvbox has "reset" and "destroy". The "reset" option only deletes arvados data (database, keep, git repos, etc) whereas the "destroy" option deletes the whole thing including the package caches. This looks like a problem in your go libraries (maybe arvbox isn't using govendor properly? not sure) you can try deleting ~/.arvbox/arvbox/gopath or using "arvbox destroy" to wipe out the whole thing.

Actions #10

Updated by Peter Amstutz over 6 years ago

  • Status changed from New to Feedback

Also I think our cwltest package needs to be updated, I believe that is why it is failing on some clusters but not others.

Actions #11

Updated by Peter Amstutz over 6 years ago

  • Status changed from Feedback to Resolved

Finally everything is blue again.

Actions #12

Updated by Tom Morris about 6 years ago

  • Release set to 17
Actions

Also available in: Atom PDF