Bug #7839

[Deployment] Python requests backports break pip

Added by Chen Chen over 6 years ago. Updated about 6 years ago.

Status:
Resolved
Priority:
High
Assigned To:
-
Category:
Deployment
Target version:
-
Start date:
11/23/2015
Due date:
% Done:

100%

Estimated time:
Story points:
-

Description

Show the installed and candidate version of python-requests and python-pip
root@shell:~# apt-cache policy python-requests
python-requests:
Installed: 2.8.1
Candidate: 2.8.1
Version table: *** 2.8.1 0
500 http://apt.arvados.org/ jessie/main amd64 Packages
100 /var/lib/dpkg/status
2.8.0 0
500 http://apt.arvados.org/ jessie/main amd64 Packages
2.7.0 0
500 http://apt.arvados.org/ jessie/main amd64 Packages
2.4.3-6 0
500 http://mirrors.163.com/debian/ jessie/main amd64 Packages
root@shell:~# apt-cache policy python-pip
python-pip:
Installed: 1.5.6-5
Candidate: 1.5.6-5
Version table: *** 1.5.6-5 0
500 http://mirrors.163.com/debian/ jessie/main amd64 Packages
100 /var/lib/dpkg/status

pip is broken
root@shell:~# pip freeze
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in <module>
load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 356, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2476, in load_entry_point
return ep.load()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load
['__name__'])
File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 74, in <module>
from pip.vcs import git, mercurial, subversion, bazaar # noqa
File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
from pip.download import path_to_url
File "/usr/lib/python2.7/dist-packages/pip/download.py", line 25, in <module>
from requests.compat import IncompleteRead
ImportError: cannot import name IncompleteRead

Get the official stocked version
root@shell:~# apt-get install python-requests=2.4.3-6
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
python-cffi python-cryptography python-ndg-httpsclient python-openssl python-ply python-pycparser python-urllib3
Suggested packages:
python-dev python-cryptography-doc python-cryptography-vectors python-openssl-doc python-openssl-dbg python-ply-doc
The following NEW packages will be installed:
python-cffi python-cryptography python-ndg-httpsclient python-openssl python-ply python-pycparser python-urllib3
The following packages will be DOWNGRADED:
python-requests
0 upgraded, 7 newly installed, 1 downgraded, 0 to remove and 0 not upgraded.
Need to get 715 kB of archives.
After this operation, 1,134 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
... SKIPPING ...

It's working now
root@shell:~# pip freeze
Pillow==2.6.1
Pygments==2.0.1
SOAPpy==0.12.22
argparse==1.2.1
arvados-fuse==0.1.20150916213104
arvados-python-client==0.1.20151024192127
cffi==0.8.6
chardet==2.3.0
ciso8601==1.0.1
colorama==0.3.2
cryptography==0.6.1
defusedxml==0.4.1
... SKIPPING ...

Upgrade to the package provided by Arvados repository
root@shell:~# apt-get install python-requests
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
python-cffi python-cryptography python-ndg-httpsclient python-openssl python-ply python-pycparser python-urllib3
Use 'apt-get autoremove' to remove them.
The following packages will be upgraded:
python-requests
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
... SKIPPING ...

Broken again
root@shell:~# pip freeze
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in <module>
load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 356, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2476, in load_entry_point
return ep.load()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load
['__name__'])
File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 74, in <module>
from pip.vcs import git, mercurial, subversion, bazaar # noqa
File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
from pip.download import path_to_url
File "/usr/lib/python2.7/dist-packages/pip/download.py", line 25, in <module>
from requests.compat import IncompleteRead
ImportError: cannot import name IncompleteRead

Associated revisions

Revision 6ca2758f (diff)
Added by Brett Smith over 6 years ago

7839: Stop backporting requests for Python 2.

This is no longer needed now that our Keep client is built on top of
pycurl. Confirmed by building all our Python packages from source
into a fresh virtualenv. requests does not appear in `pip freeze`.

requests is still needed for Python 3 because docker-py uses it.

Closes #7839.

Revision 6ca2758f (diff)
Added by Brett Smith over 6 years ago

7839: Stop backporting requests for Python 2.

This is no longer needed now that our Keep client is built on top of
pycurl. Confirmed by building all our Python packages from source
into a fresh virtualenv. requests does not appear in `pip freeze`.

requests is still needed for Python 3 because docker-py uses it.

Closes #7839.

History

#1 Updated by Chen Chen over 6 years ago

Sorry I forgot to preview before submission. Please forgive the ugly format.

#2 Updated by Brett Smith over 6 years ago

  • Target version set to Arvados Future Sprints

#3 Updated by Brett Smith over 6 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados-dev|commit:6ca2758f31fb1a4afeb4eb5aca3cd8d6af402169.

#4 Updated by Brett Smith over 6 years ago

  • Subject changed from Package "python-requests" shipped in Arvados Debian Jessie repo broke python-pip to [Deployment] Python requests backports break pip
  • Category set to Deployment
  • Status changed from Resolved to New
  • % Done changed from 100 to 0

Chen,

Thanks for reporting this. As it happens, no Arvados tools currently use requests, so it's safe to downgrade and pin the package. I've updated our package scripts to stop building it, and I'll revoke what we currently have published at the first opportunity.

Unfortunately, this issue still exists for python3-requests, which we need to backport in order to accommodate one of our other dependencies, docker-py. Figuring out how to fix that will take more investigation. Hopefully this is still an improvement for you, since you'll only need the backported packages on Arvados compute nodes, where you're less likely to need to run pip interactively.

#5 Updated by Brett Smith over 6 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:6ca2758f31fb1a4afeb4eb5aca3cd8d6af402169.

#6 Updated by Brett Smith about 6 years ago

  • Target version deleted (Arvados Future Sprints)

Also available in: Atom PDF