Bug #3857
Updated by Tim Pierce over 9 years ago
Executive summary: If an arvados.KeepClient is connecting Change the connect timeout for the Python KeepClient to a Keep proxy (service_type == 'proxy'): * The TCP connection timeout should be 2 or 3 seconds. * Otherwise, connection timeout should be 20 seconds. Leave all other timeouts as they are. More in detail: Right now the Python KeepClient takes a single timeout setting when it's instantiated, and uses that for all HTTP requests by passing it along when it instantiates its own httplib2.Http objects. It seems like httplib2.Http is using this same timeout for all aspects of the request. However, we'd like to be smarter than that: * The initial attempt to connect, to send the request, should use a short timeout. * After the request is sent, it's appropriate to use a longer timeout. (The worst-case scenario is that we're PUTting 64MiB to a proxy. Just sending the data will take a little while, and then we'll have to wait for it to be duplexed to several servers before we receive any response.) With the single timeout we currently have, a small value causes premature failures using tools like arv-put (see #3846), but a long value can cause clients to spend needless time waiting to contact a Keep service that's down. Investigate whether it's possible to set different timeouts for these different operations, and implement it if so. Without looking, this might be the difference between the socket module's timeout and httplib2.Http's, or httplib2.Http might accept different configuration arguments for these.