Project

General

Profile

Actions

Bug #6651

closed

"This website is under heavy load" error causes a parsing error

Added by Nico César over 8 years ago. Updated over 4 years ago.

Status:
Duplicate
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Story points:
-

Description

/usr/local/arvados/install-arvados-tokens.rb production

Error: 757: unexpected token at '<h1>This website is under heavy load</h1><p>We're sorry, too many people are accessing this website at the same time. We're working on this problem. Please try again later.</p>'
/usr/local/rvm/gems/ruby-2.1.4/gems/json-1.8.2/lib/json/common.rb:155:in `parse'
/usr/local/rvm/gems/ruby-2.1.4/gems/json-1.8.2/lib/json/common.rb:155:in `parse'
/usr/local/rvm/gems/ruby-2.1.4/gems/arvados-0.1.20150615153458/lib/arvados.rb:214:in `api_exec'
/usr/local/rvm/gems/ruby-2.1.4/gems/arvados-0.1.20150615153458/lib/arvados.rb:83:in `block (4 levels) in initialize'
/usr/local/arvados/install-arvados-tokens.rb:81:in `<main>'
<pre>

passenger default of overloaded is in HTML not in json
Actions #1

Updated by Joshua Randall about 8 years ago

I just observed this on our system as well - when the API server is overloaded, workbench ends up putting up a generic error about parsing rather than any message about the heavy load.

Actions #2

Updated by Joshua Randall about 8 years ago

I have worked around this problem with the following nginx config:

in nginx.conf:

    passenger_request_queue_overflow_status_code 513;
    passenger_intercept_errors on;

in cond.d/arvados-api.conf, inside the "server { listen 127.0.0.1:8000 ... }" block:

  error_page 513 =503 /heavyload.json;

In /var/www/arvados-api/current/public/heavyload.json:

{"errors":["This website is under heavy load (queue full)"]}

I change the status code for the "heavy load" situation within passenger from 503 to 513 (since there are other situations that could also cause 503 errors). I then configure nginx to process the 513 status by returning the /heavyload.json file instead of the default handler, and also recodes the status back to 503.

Actions #3

Updated by Tom Morris over 6 years ago

  • Assigned To set to Nico César
  • Target version set to 2017-08-16 sprint
Actions #4

Updated by Nico César over 6 years ago

  • Target version changed from 2017-08-16 sprint to Arvados Future Sprints

this is a duplicate

Actions #5

Updated by Nico César almost 5 years ago

  • Status changed from New to Duplicate
Actions #6

Updated by Tom Morris over 4 years ago

  • Assigned To deleted (Nico César)
  • Target version deleted (Arvados Future Sprints)
Actions

Also available in: Atom PDF