[SDK] PyCurl 7.21.5 breaks keepclient
|Assignee:||Tom Clegg||% Done:|
|Target version:||2017-04-12 sprint|
|Story points||-||Remaining (hours)||0.00 hour|
|Velocity based estimate||-|
Symptom: keepclient fails with
TypeError: _socket_open() takes at least 3 arguments (2 given)
This is due to an API change in PyCurl:
Version 7.21.5 [requires libcurl-7.19.0 or better] - 2016-01-05 * Breaking change: OPENSOCKETFUNCTION callback API now mirrors that of libcurl: 1. The callback now takes two arguments, `purpose' and `address`. Previously the callback took `family', `socktype', `protocol` and `addr' arguments. 2. The second argument to the callback, `address', is a `namedtuple' with `family', `socktype', `protocol' and `addr' fields. 3. `addr' field on `address' for AF_INET6 family addresses is a 4-tuple of (address, port, flow info, scope id) which matches Python's `socket.getaddrinfo' API. It seems that libcurl may mishandle error return from an opensocket callback, as would happen when code written for pre-PycURL 7.21.5 API is run with PycURL 7.21.5 or newer, resulting in the application hanging.
As a quick fix, we can add a version constraint
<7.21.5 to prevent installing the newest PyCurl package.
Add pycurl version constraint to Python SDK because of breaking API change, refs #8104
#2 Updated by Peter Amstutz 3 months ago
Suggest the following version dependency change:
- 'pycurl >=18.104.22.168, <7.21.5', + 'pycurl >=7.19.3',
Due to the API change, we need to detect the pycurl version and use the correct API.
Specifying >=7.19.3 allows us to depend on OS-provided packages on Debian 8, Ubuntu 14.04, Ubuntu 16.04.
Will need to provide backport for Centos 7 (which ships 7.19.0 which we can't use due to #6073).
Because pycurl 7.21.5 has a breaking API change, we should continue backport 22.214.171.124 to avoid breaking anything else that might depend on pycurl.
Debian 7, Centos 6, and Ubuntu 12.04 would also require backports, but they are EOL.
#4 Updated by Peter Amstutz 3 months ago
(I think this is the minimum version that has the #6073 fix: https://github.com/pycurl/pycurl/blob/master/ChangeLog#L473)
#10 Updated by Peter Amstutz 3 months ago
(On further discussion, disregard previous note)
What's the reasoning here? Explicitly closing the socket after every request seems like it will defeat HTTP keepalive.
try: curl.perform() except Exception as e: raise arvados.errors.HttpError(0, str(e)) finally: if self._socket: self._socket.close() self._socket = None