Project

General

Profile

Actions
Copy link

Bug #3783

closed

[Crunch] 'arv pipeline run --run-here ...' failed at end even though all jobs/tasks completed successfully

Added by Abram Connelly over 9 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
Low
Assigned To:
Brett Smith
Category:
Crunch
Target version:
-
Story points:
-

Description

When running pipeline instance qr1hi-d1hrv-9jrnsp198ztr0kn, I get the following error after issuing an 'arv pipeline run --run-here ...' command:

2014-08-31 09:13:52 +0000 -- pipeline_instance qr1hi-d1hrv-9jrnsp198ztr0kn
ChopGFF                     qr1hi-8i9sb-g4lrf35jmn7dleq d071779966740d0f4884c17c3712a6a6+8369520
CreateChoppedGFFDirList     qr1hi-8i9sb-98eoeaborr1jlrw c26af40291d10bb6f0d94edd349be41d+64
CreateTileSetFromChoppedGFF qr1hi-8i9sb-zyva8qlbfmjsnse {:done=>176, :running=>0, :failed=>0, :todo=>0}

2014-08-31 09:14:03 +0000 -- pipeline_instance qr1hi-d1hrv-9jrnsp198ztr0kn
ChopGFF                     qr1hi-8i9sb-g4lrf35jmn7dleq d071779966740d0f4884c17c3712a6a6+8369520
CreateChoppedGFFDirList     qr1hi-8i9sb-98eoeaborr1jlrw c26af40291d10bb6f0d94edd349be41d+64
CreateTileSetFromChoppedGFF qr1hi-8i9sb-zyva8qlbfmjsnse {:done=>176, :running=>0, :failed=>0, :todo=>0}

2014-08-31 09:14:13 +0000 -- pipeline_instance qr1hi-d1hrv-9jrnsp198ztr0kn
ChopGFF                     qr1hi-8i9sb-g4lrf35jmn7dleq d071779966740d0f4884c17c3712a6a6+8369520
CreateChoppedGFFDirList     qr1hi-8i9sb-98eoeaborr1jlrw c26af40291d10bb6f0d94edd349be41d+64
CreateTileSetFromChoppedGFF qr1hi-8i9sb-zyva8qlbfmjsnse {:done=>176, :running=>0, :failed=>0, :todo=>0}
arv-run-pipeline-instance 23798: names:  Personal Tile Set Pipeline (Batches at a time)
arv-run-pipeline-instance 23798: Creating collection {:owner_uuid=>"qr1hi-tpzed-rd7cyyspidta11c", :name=>"Output of CreateTileSetFromChoppedGFF of Personal Tile Set Pipeline (Batches at a time) at 2014-08-31T09:14:16Z", :portable_data_hash=>nil, :manifest_text=>""}
/usr/local/rvm/gems/ruby-2.1.1/gems/arvados-0.1.20140829123712/lib/arvados.rb:243:in `api_exec': ["#<ActiveRecord::UnknownAttributeError: unknown attribute: name>"] (Arvados::TransactionFailedError)
        from /usr/local/rvm/gems/ruby-2.1.1/gems/arvados-0.1.20140829123712/lib/arvados.rb:82:in `block (4 levels) in initialize'
        from /usr/local/rvm/gems/ruby-2.1.1/gems/arvados-cli-0.1.20140829123712/bin/arv-run-pipeline-instance:588:in `block in run'
        from /usr/local/rvm/gems/ruby-2.1.1/gems/arvados-cli-0.1.20140829123712/bin/arv-run-pipeline-instance:501:in `each'
        from /usr/local/rvm/gems/ruby-2.1.1/gems/arvados-cli-0.1.20140829123712/bin/arv-run-pipeline-instance:501:in `run'
        from /usr/local/rvm/gems/ruby-2.1.1/gems/arvados-cli-0.1.20140829123712/bin/arv-run-pipeline-instance:748:in `<top (required)>'
        from /usr/local/rvm/gems/ruby-2.1.1/bin/arv-run-pipeline-instance:23:in `load'
        from /usr/local/rvm/gems/ruby-2.1.1/bin/arv-run-pipeline-instance:23:in `<main>'
        from /usr/local/rvm/gems/ruby-2.1.1/bin/ruby_executable_hooks:15:in `eval'
        from /usr/local/rvm/gems/ruby-2.1.1/bin/ruby_executable_hooks:15:in `<main>'

The last job in the pipeline is still marked as 'running' in workbench.

Actions
Copy link
 #1

Updated by Brett Smith over 9 years ago

  • Subject changed from 'arv pipeline run --run-here ...' failed at end even though all jobs/tasks completed successfully to [Crunch] 'arv pipeline run --run-here ...' failed at end even though all jobs/tasks completed successfully
  • Category set to Crunch

At first glance, it looks like the CLI SDK in new enough to try to use name attributes rather than name links. I think that change still hasn't been deployed to qr1hi, so that could cause a crash if arv-run-pipeline-instance doesn't have code to fall back.

Actions
Copy link
 #2

Updated by Ward Vandewege over 9 years ago

  • Priority changed from Normal to Low
Actions
Copy link
 #3

Updated by Ward Vandewege over 9 years ago

  • Priority changed from Low to Normal
Actions
Copy link
 #4

Updated by Ward Vandewege over 9 years ago

  • Priority changed from Normal to Low
Actions
Copy link
 #5

Updated by Brett Smith over 9 years ago

  • Status changed from New to Resolved
  • Assigned To set to Brett Smith

Abram,

I confirmed that the problem was that lightning-dev2 had client software that was newer than the API server. As we discussed, I downgraded that software to be compatible with the current API server.

Commands that update your installed Ruby gems, like gem update, can cause this sort of out-of-sync problem. I'm not sure if you've been dealing with gems as part of your development work, but if so, please be mindful of this potential side effect.

Actions
Copy link

Also available in: Atom PDF