Arvados

Just to keep the record of what we already chatted:

Tried to run the conformance tests on arvbox and got errors similar to:

Test [13/127] 
Test failed: /home/lucas/venv/bin/arvados-cwl-runner --api=containers --compute-checksum --disable-reuse --outdir=/tmp/tmp0wBPwn --quiet v1.0/cat4-tool.cwl v1.0/cat-job.json
Test command execution in Docker with stdin and stdout redirection
Returned non-zero
2018-03-02 16:04:26 cwltool ERROR: Workflow error, try again with --debug for more information:
v1.0/cat4-tool.cwl:7:5: 'module' object has no attribute 'get_image'

There are also some tests failing, don't know if they're related: https://ci.curoverse.com/job/developer-run-tests/626/

Thanks.

Updated with fix 13140-cwl-conformance @ cde5fc115a5d543eb6522fdad6e651dfbe857656

I'm having problems when trying to run the conformance tests on arvbox:

(venv) lucas@veritas:~/common-workflow-language$ ./run_test.sh RUNNER=arvados-cwl-runner EXTRA="--api=containers --compute-checksum --disable-reuse" 
--- Running conformance test v1.0 on /home/lucas/venv/bin/arvados-cwl-runner ---
/home/lucas/venv/bin/arvados-cwl-runner 1.0.20180302211217, arvados-python-client 0.1.20180223161544, cwltool 1.0.20180225105849
Test [1/127] 
Test timed out: /home/lucas/venv/bin/arvados-cwl-runner --api=containers --compute-checksum --disable-reuse --outdir=/tmp/tmp_fJ2Wz --quiet v1.0/bwa-mem-tool.cwl v1.0/bwa-mem-job.json
General test of command line generation
Test [2/127] 

Maybe it's a local issue, I tried resetting the arvbox instance from scratch, re-building the image, but it's not working for me. On workbench the jobs appear as Queued and are kept that way after the timeout.

As suggested by Peter I asked for the crunch-dispatch-local logs and I'm seeing this error:

[...]
2018-03-05_19:01:10.61051 # git.curoverse.com/arvados.git/services/crunch-run
2018-03-05_19:01:10.61054 src/git.curoverse.com/arvados.git/services/crunch-run/crunchrun.go:1777: cannot use docker (type *client.Client) as type ThinDockerClient in argument to NewContaine
rRunner:
2018-03-05_19:01:10.61055       *client.Client does not implement ThinDockerClient (wrong type for ContainerAttach method)
2018-03-05_19:01:10.61055               have ContainerAttach("github.com/docker/docker/vendor/golang.org/x/net/context".Context, string, types.ContainerAttachOptions) (types.HijackedResponse
, error)
2018-03-05_19:01:10.61055               want ContainerAttach("golang.org/x/net/context".Context, string, types.ContainerAttachOptions) (types.HijackedResponse, error)
[...]

Lucas Di Pentima wrote:

As suggested by Peter I asked for the crunch-dispatch-local logs and I'm seeing this error:

[...]

arvbox has "reset" and "destroy". The "reset" option only deletes arvados data (database, keep, git repos, etc) whereas the "destroy" option deletes the whole thing including the package caches. This looks like a problem in your go libraries (maybe arvbox isn't using govendor properly? not sure) you can try deleting ~/.arvbox/arvbox/gopath or using "arvbox destroy" to wipe out the whole thing.