Bug #19126
closed
Disable nginx proxy caching for controller
Added by Ward Vandewege almost 2 years ago.
Updated almost 2 years ago.
Release relationship:
Auto
Description
When deployed behind nginx for tls termination, controller's responses can get truncated when nginx's reverse proxy tries to cache responses which exceed its temp file size. We don't really need nginx to do any caching in this scenario.
- update documentation accordingly
- update the arvados-server boot template (also used in our test suite)
- update our internal salt configs
- Status changed from New to In Progress
- Description updated (diff)
- Subject changed from Disable nginx proxy caching on controller to Disable nginx proxy caching for controller
- Description updated (diff)
The doc update refers to large responses, but the proxy_request_buffering setting is about large request bodies. Should we add "proxy_buffering off" as well?
Also, nginx doc says "When HTTP/1.1 chunked transfer encoding is used to send the original request body, the request body will be buffered regardless of the directive value unless HTTP/1.1 is enabled for proxying." ...and the default is HTTP/1.0. So I think we need to add "proxy_http_version 1.1;" too.
Tom Clegg wrote:
Also, nginx doc says "When HTTP/1.1 chunked transfer encoding is used to send the original request body, the request body will be buffered regardless of the directive value unless HTTP/1.1 is enabled for proxying." ...and the default is HTTP/1.0. So I think we need to add "proxy_http_version 1.1;" too.
As we discussed, the thing that actually works around the problem here is disabling the use of temporary files in response buffering, i.e. the proxy_max_temp_file_size 0; line.
So it seems like we should either limit this change to that line, or go the whole way and disable buffering in every direction:
proxy_max_temp_file_size 0;
proxy_request_buffering off;
proxy_buffering off;
proxy_http_version 1.1;
I think we should go all the way and disable buffering in both directions. I don't think we get any benefit from buffering -- afaict all it is is a source of problems.
LGTM.
Seems to be based on the #16345 branch, but I've just merged that now, so I don't think it needs to be rebased.
- % Done changed from 0 to 100
- Status changed from In Progress to Resolved
Applied in changeset arvados-private:commit:arvados|91dc80d9060c5aeb04228f0f280ecc3db182ff94.
Also available in: Atom
PDF
Updated by 
Updated by 
Updated by