Idea #20602
Updated by Peter Amstutz 11 months ago
Use Origin header to distinguish requests coming from workbench2 (list of "interactive" origins should be configurable to accommodate future applications that are interactive and talk to API server).
Traffic from interactive applications (workbench2) is prioritized, we want to avoid returning API errors and return responses as quickly as possible, at the expense of "general admission" requests from non-interactive API clients and services.
Keep-web should also use/pass through Origin header to API to indicate when a request is coming from workbench2 and should be handled at interactive priority (i.e. when getting a collection listing).
To reduce queuing time the "general admission" queue should be small provide backpressure with 503 responses, because they are assumed to be fine to wait and retry on 503.
Then the UI queue (higher priority) can be a bit larger to avoid API errors, but by having the general admission queue be smaller, the UI requests avoid being stuck behind a large queue of general admission requests.
Something like
40 workers
general admission queue is 2x = 80 -> gets 503 errors much sooner
prioritized is 4x = 160 (or higher)
alternately --
As each connection comes in, read headers and assign a priority, put in a priority queue
Rework how controller forwards connections to passenger/rails
Limit concurrent forwarded requests to Rails == worker pool
As a worker becomes free, pull the next request from the priority queue -- UI requests will sort before general admission requests.
Also - need to look at passenger behavior & settings related to priority and detecting when workers are busy.