Project

General

Profile

Actions

Support #9628

closed

CWL Tutorial Does not Run

Added by Adam Savitzky over 7 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Tom Morris
Category:
-
Target version:
-
Due date:
Story points:
-

Description

I'm working through the example here: http://doc.arvados.org/user/cwl/cwl-runner.html using curoverse cloud and the arvados-cwl-runner.

Upon running:

arvados-cwl-runner bwa-mem.cwl bwa-mem-input.yml

The pipeline fails with the error:

arvados.cwl-runner[14764] ERROR: Caught unhandled exception, marking pipeline as failed.  Error was: "filename 'sha256:b587749c9f340508910ac4137bc54b367ad42537b934882f3e6555ee0cdd02bb/json' not found" 

Here is the full output (ran with --debug):

> arvados-cwl-runner --debug bwa-mem.cwl bwa-mem-input.yml
/usr/local/bin/arvados-cwl-runner 1.0.20160713211920, arvados-python-client 0.1.20160713191718, cwltool 1.0.20160712154127
2016-07-18 12:57:35 arvados.cwl-runner[14933] INFO: Uploading Docker image biodckr/bwa
1143M / 1143M 100.0% 
Collection saved as 'Docker image biodckr/bwa:latest sha256:b5877'
qr1hi-4zz18-9wm2lvmp9p0x09f
2016-07-18 13:00:54 arvados.cwl-runner[14933] ERROR: Caught unhandled exception, marking pipeline as failed.  Error was: "filename 'sha256:b587749c9f340508910ac4137bc54b367ad42537b934882f3e6555ee0cdd02bb/json' not found" 
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/arvados_cwl/__init__.py", line 184, in arvExecutor
    runnable.run(**kwargs)
  File "/usr/local/lib/python2.7/site-packages/arvados_cwl/arvjob.py", line 216, in run
    job_spec = self.arvados_job_spec(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/arvados_cwl/arvjob.py", line 202, in arvados_job_spec
    workflowmapper = super(RunnerJob, self).arvados_job_spec(dry_run=dry_run, pull_image=pull_image, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/arvados_cwl/runner.py", line 47, in arvados_job_spec
    self.upload_docker(self.tool)
  File "/usr/local/lib/python2.7/site-packages/arvados_cwl/runner.py", line 40, in upload_docker
    arv_docker_get_image(self.arvrunner.api, docker_req, True, self.arvrunner.project_uuid)
  File "/usr/local/lib/python2.7/site-packages/arvados_cwl/arvdocker.py", line 32, in arv_docker_get_image
    arvados.commands.keepdocker.main(args, stdout=sys.stderr)
  File "/usr/local/lib/python2.7/site-packages/arvados/commands/keepdocker.py", line 402, in main
    json_file = image_tar.extractfile(image_tar.getmember(image_hash + '/json'))
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.py", line 1827, in getmember
    raise KeyError("filename %r not found" % name)
KeyError: "filename 'sha256:b587749c9f340508910ac4137bc54b367ad42537b934882f3e6555ee0cdd02bb/json' not found" 
Workflow error, try again with --debug for more information:
  Workflow did not return a result.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/cwltool/main.py", line 707, in main
    **vars(args))
  File "/usr/local/lib/python2.7/site-packages/arvados_cwl/__init__.py", line 216, in arvExecutor
    raise WorkflowException("Workflow did not return a result.")
WorkflowException: Workflow did not return a result.

Related issues

Related to Arvados - Idea #8568: [SDKs] `arv keep docker` supports Docker 1.10+ResolvedRadhika Chippada02/26/2016Actions
Actions #1

Updated by Brett Smith over 7 years ago

  • Tracker changed from Bug to Support
  • Status changed from New to In Progress
  • Priority changed from High to Normal

Adam,

As discussed on IRC, you're apparently hitting #8568: our code that uploads Docker images to Arvados doesn't know how to handle the image format used by Docker 1.10+. If you run on a system with Docker 1.9, you should get past this. Sorry for the trouble.

Actions #2

Updated by Tom Morris over 7 years ago

  • Assigned To set to Tom Morris

Adam - Did Brett's suggestion fix this for you? Can we mark this as resolved?

Actions #3

Updated by Tom Morris over 7 years ago

  • Status changed from In Progress to Resolved

Marking as resolved. Please reopen if you're still having issues.

Actions

Also available in: Atom PDF