Bug #6651
closed"This website is under heavy load" error causes a parsing error
Description
/usr/local/arvados/install-arvados-tokens.rb production
Error: 757: unexpected token at '<h1>This website is under heavy load</h1><p>We're sorry, too many people are accessing this website at the same time. We're working on this problem. Please try again later.</p>' /usr/local/rvm/gems/ruby-2.1.4/gems/json-1.8.2/lib/json/common.rb:155:in `parse' /usr/local/rvm/gems/ruby-2.1.4/gems/json-1.8.2/lib/json/common.rb:155:in `parse' /usr/local/rvm/gems/ruby-2.1.4/gems/arvados-0.1.20150615153458/lib/arvados.rb:214:in `api_exec' /usr/local/rvm/gems/ruby-2.1.4/gems/arvados-0.1.20150615153458/lib/arvados.rb:83:in `block (4 levels) in initialize' /usr/local/arvados/install-arvados-tokens.rb:81:in `<main>' <pre> passenger default of overloaded is in HTML not in json
Updated by Joshua Randall about 8 years ago
I just observed this on our system as well - when the API server is overloaded, workbench ends up putting up a generic error about parsing rather than any message about the heavy load.
Updated by Joshua Randall about 8 years ago
I have worked around this problem with the following nginx config:
in nginx.conf:
passenger_request_queue_overflow_status_code 513; passenger_intercept_errors on;
in cond.d/arvados-api.conf, inside the "server { listen 127.0.0.1:8000 ... }" block:
error_page 513 =503 /heavyload.json;
In /var/www/arvados-api/current/public/heavyload.json:
{"errors":["This website is under heavy load (queue full)"]}
I change the status code for the "heavy load" situation within passenger from 503 to 513 (since there are other situations that could also cause 503 errors). I then configure nginx to process the 513 status by returning the /heavyload.json file instead of the default handler, and also recodes the status back to 503.
Updated by Tom Morris over 6 years ago
- Assigned To set to Nico César
- Target version set to 2017-08-16 sprint
Updated by Nico César over 6 years ago
- Target version changed from 2017-08-16 sprint to Arvados Future Sprints
this is a duplicate
Updated by Tom Morris over 4 years ago
- Assigned To deleted (
Nico César) - Target version deleted (
Arvados Future Sprints)