Bug #3031
closed
[Crunch] crash when trying to re-run job (docker image not found)
Added by Ward Vandewege about 10 years ago. Updated over 4 years ago.
Description
Need to figure out what behavior is desired (if different from the current behavior) when submitting a job with a docker image that has not yet been stored in Keep.
Updated by Ward Vandewege about 10 years ago
Started POST "/jobs" for 207.172.212.131 at 2014-06-17 13:37:30 +0000
Processing by JobsController#create as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"CbbmfoeLpMAXMuFRf7LRro4KFEcFTujnAaHuW1Tazw0=", "job"=>{"script"=>"stuff.py", "script_version"=>"13db72223aa80bce731a9b33cb1dec2bc5ccdae9", "repository"=>"peter", "output_is_persistent"=>"false", "supplied_script_version"=>"13db72223aa80bce731a9b33cb1dec2bc5ccdae9", "nondeterministic"=>"", "script_parameters"=>"{}", "runtime_constraints"=>"{\"docker_image\":\"arvados/jobs\"}"}, "button"=>""}
#<RuntimeError: Docker image locator Docker image not found [API: 422]>
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/models/arvados_api_client.rb:107:in `api'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/models/arvados_base.rb:176:in `save'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:206:in `create'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.1/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.1/lib/abstract_controller/base.rb:189:in `process_action'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.1/lib/action_controller/metal/rendering.rb:10:in `process_action'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.1/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:113:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:113:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:299:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:435:in `thread_with_optional_api_token'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:424:in `block in make_lambda'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:298:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:298:in `block in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:282:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:282:in `block (2 levels) in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:418:in `block in thread_with_mandatory_api_token'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:393:in `thread_with_api_token'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:416:in `thread_with_mandatory_api_token'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:424:in `block in make_lambda'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:281:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:281:in `block in halting_and_conditional'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:299:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:299:in `block (2 levels) in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/app/controllers/application_controller.rb:349:in `thread_clear'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:424:in `block in make_lambda'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:298:in `call'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:298:in `block in halting'
/var/www/workbench.4xphq.arvadosapi.com/releases/20140616194234/vendor/bundle/ruby/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:229:in `call'
...
Updated by Brett Smith about 10 years ago
This is sort of churn caused by our move from dynamically installing Docker images with docker pull, to storing them in Keep. You're trying to re-run a Job that was written for the old system, and now it fails because the image isn't available in Keep.
(Actually, I bet it would work now, because I installed the arvados/jobs image in Keep earlier this week for my own testing. But this is definitely why the original crash happened.)
Still, this error could come up today if somebody tried to run a Job without installing the Docker image first, or the image Collection or its associated Links were deleted. The exception is more or less expected; I'm guessing you want the Jobs controller to catch this and render a nicer/quieter error?
Updated by Tom Clegg about 10 years ago
- Subject changed from crash when trying to re-run job (docker image not found) to [Crunch] crash when trying to re-run job (docker image not found)
Updated by Tom Clegg about 10 years ago
- Target version changed from 2014-08-06 Sprint to Arvados Future Sprints
Updated by Tom Clegg about 10 years ago
- Target version deleted (
Arvados Future Sprints)
Updated by