[Crunch] google-api-python-client fails to install
From qr1hi-8i9sb-wleo6rt176ublrc, using docker image bcosc/all-tools
1/7/2015 1:20:27 PM compute20 1 task-print 0 File "/tmp/crunch-job/src/crunch_scripts/bwa-index", line 3, in <module>
1/7/2015 1:20:27 PM compute20 1 task-print 0 import arvados
1/7/2015 1:20:27 PM compute20 1 task-print 0 File "/usr/local/lib/python2.7/dist-packages/arvados/__init__.py", line 21, in <module>
1/7/2015 1:20:27 PM compute20 1 task-print 0 from api import *
1/7/2015 1:20:27 PM compute20 1 task-print 0 File "/usr/local/lib/python2.7/dist-packages/arvados/api.py", line 8, in <module>
1/7/2015 1:20:27 PM compute20 1 task-print 0 import apiclient
1/7/2015 1:20:27 PM compute20 1 task-print 0 File "/usr/local/lib/python2.7/dist-packages/apiclient/__init__.py", line 9, in <module>
1/7/2015 1:20:27 PM compute20 1 task-print 0 'Previous version of google-api-python-client detected; due to a '
1/7/2015 1:20:27 PM compute20 1 task-print 0 RuntimeError: Previous version of google-api-python-client detected; due to a packaging issue, we cannot perform an in-place upgrade. To repair, remove and reinstall this package, along with oauth2client and uritemplate. One can do this with pip via
1/7/2015 1:20:27 PM compute20 1 task-print 0 pip install -I google-api-python-client
1/7/2015 1:20:27 PM compute20 1 task-dispatch 0 srun: error: compute20: task 0: Exited with exit code 1
#4 Updated by Tim Pierce over 5 years ago
Tom Clegg wrote:
(This is probably fallout from changing the version format from worked-for-a-while 0.1.date.commit to PEP440-compliant 0.1.date+commit.)
It's not the arvados-python-client version number that it's whinging about, it's google-api-python-client. (Or do you mean that because we're using a properly formatted version number now for Arvados, it's better able to handle client dependencies and therefore exposing cases like this that would not have otherwise seen daylight? I guess something like that is possible?)
Either way -- I can't think of any reason I wouldn't use
pip install -I any time I encounter this by hand, so I can't see any reason not to make that the default.
#11 Updated by Brett Smith about 5 years ago
- Status changed from New to Resolved
The original job didn't specify an arvados_sdk_version, so changing crunch-job's install process for that would not change anything. My reproducing job did specify an arvados_sdk_version, but
pip -I was still a noop because the Docker image doesn't include virtualenv.
The problem is simply that the Docker image has both the
googleapiclient modules installed. As long as that's true, this exception will be raised on import. There's nothing we can do about it on the Crunch level.
Bryan, I believe you've effectively worked around this by adding virtualenv to the image and using
arvados_sdk_version, but next time you're in there, you might want to clean up by removing the old
apiclient Python module (
pip uninstall apiclient) in favor of
googleapiclient. Let me know if you have any questions about it.