Bug #3031

[Crunch] crash when trying to re-run job (docker image not found)

Added by Ward Vandewege over 5 years ago. Updated about 5 years ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
1.0

Description

Need to figure out what behavior is desired (if different from the current behavior) when submitting a job with a docker image that has not yet been stored in Keep.

Associated revisions

Revision 24e88096 (diff)
Added by Brett Smith over 5 years ago

3031: Improve "Docker image not found" error message.

Rails automatically adds the column name to rendered error message, so
the old version was coming out as "Docker image locator Docker image
not found." This version reads better. Refs #3031.

History

#1 Updated by Ward Vandewege over 5 years ago

Started POST "/jobs" for 207.172.212.131 at 2014-06-17 13:37:30 +0000
Processing by JobsController#create as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"CbbmfoeLpMAXMuFRf7LRro4KFEcFTujnAaHuW1Tazw0=", "job"=>{"script"=>"stuff.py", "script_version"=>"13db72223aa80bce731a9b33cb1dec2bc5ccdae9", "repository"=>"peter", "output_is_persistent"=>"false", "supplied_script_version"=>"13db72223aa80bce731a9b33cb1dec2bc5ccdae9", "nondeterministic"=>"", "script_parameters"=>"{}", "runtime_constraints"=>"{\"docker_image\":\"arvados/jobs\"}"}, "button"=>""}
#<RuntimeError: Docker image locator Docker image not found [API: 422]>
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/models/arvados_api_client.rb:107:in `api'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/models/arvados_base.rb:176:in `save'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:206:in `create'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.1/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.1/lib/abstract_controller/base.rb:189:in `process_action'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.1/lib/action_controller/metal/rendering.rb:10:in `process_action'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.1/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:113:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:113:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:299:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:435:in `thread_with_optional_api_token'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:424:in `block in make_lambda'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:298:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:298:in `block in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:282:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:282:in `block (2 levels) in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:418:in `block in thread_with_mandatory_api_token'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:393:in `thread_with_api_token'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:416:in `thread_with_mandatory_api_token'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:424:in `block in make_lambda'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:281:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:281:in `block in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:299:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:349:in `thread_clear'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:424:in `block in make_lambda'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:298:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:298:in `block in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:229:in `call'
...

#2 Updated by Brett Smith over 5 years ago

This is sort of churn caused by our move from dynamically installing Docker images with docker pull, to storing them in Keep. You're trying to re-run a Job that was written for the old system, and now it fails because the image isn't available in Keep.

(Actually, I bet it would work now, because I installed the arvados/jobs image in Keep earlier this week for my own testing. But this is definitely why the original crash happened.)

Still, this error could come up today if somebody tried to run a Job without installing the Docker image first, or the image Collection or its associated Links were deleted. The exception is more or less expected; I'm guessing you want the Jobs controller to catch this and render a nicer/quieter error?

#3 Updated by Tom Clegg over 5 years ago

  • Target version set to 2014-08-06 Sprint

#4 Updated by Tom Clegg over 5 years ago

  • Story points set to 1.0

#5 Updated by Tom Clegg over 5 years ago

  • Subject changed from crash when trying to re-run job (docker image not found) to [Crunch] crash when trying to re-run job (docker image not found)

#6 Updated by Tom Clegg about 5 years ago

  • Description updated (diff)

#7 Updated by Tom Clegg about 5 years ago

  • Target version changed from 2014-08-06 Sprint to Arvados Future Sprints

#8 Updated by Tom Clegg about 5 years ago

  • Target version deleted (Arvados Future Sprints)

Also available in: Atom PDF