Bug #4922
closed
[Crunch] google-api-python-client fails to install
Description
From qr1hi-8i9sb-wleo6rt176ublrc, using docker image bcosc/all-tools
1/7/2015 1:20:27 PM compute20 1 task-print 0 File "/tmp/crunch-job/src/crunch_scripts/bwa-index", line 3, in <module>
1/7/2015 1:20:27 PM compute20 1 task-print 0 import arvados
1/7/2015 1:20:27 PM compute20 1 task-print 0 File "/usr/local/lib/python2.7/dist-packages/arvados/__init__.py", line 21, in <module>
1/7/2015 1:20:27 PM compute20 1 task-print 0 from api import *
1/7/2015 1:20:27 PM compute20 1 task-print 0 File "/usr/local/lib/python2.7/dist-packages/arvados/api.py", line 8, in <module>
1/7/2015 1:20:27 PM compute20 1 task-print 0 import apiclient
1/7/2015 1:20:27 PM compute20 1 task-print 0 File "/usr/local/lib/python2.7/dist-packages/apiclient/__init__.py", line 9, in <module>
1/7/2015 1:20:27 PM compute20 1 task-print 0 'Previous version of google-api-python-client detected; due to a '
1/7/2015 1:20:27 PM compute20 1 task-print 0 RuntimeError: Previous version of google-api-python-client detected; due to a packaging issue, we cannot perform an in-place upgrade. To repair, remove and reinstall this package, along with oauth2client and uritemplate. One can do this with pip via
1/7/2015 1:20:27 PM compute20 1 task-print 0 pip install -I google-api-python-client
1/7/2015 1:20:27 PM compute20 1 task-dispatch 0 srun: error: compute20: task 0: Exited with exit code 1
Updated by Tim Pierce over 9 years ago
- Status changed from New to In Progress
- Assigned To set to Tim Pierce
- Target version set to 2015-01-28 Sprint
- Story points set to 0.5
Updated by Tom Clegg over 9 years ago
Do we just need to add "-I" here? source:sdk/cli/bin/crunch-job#L1849
shell_or_die("$venv_dir/bin/pip", "--quiet", "install", $python_src);
Updated by Tom Clegg over 9 years ago
(This is probably fallout from changing the version format from worked-for-a-while 0.1.date.commit to PEP440-compliant 0.1.date+commit.)
Updated by Tim Pierce over 9 years ago
Tom Clegg wrote:
(This is probably fallout from changing the version format from worked-for-a-while 0.1.date.commit to PEP440-compliant 0.1.date+commit.)
It's not the arvados-python-client version number that it's whinging about, it's google-api-python-client. (Or do you mean that because we're using a properly formatted version number now for Arvados, it's better able to handle client dependencies and therefore exposing cases like this that would not have otherwise seen daylight? I guess something like that is possible?)
Either way -- I can't think of any reason I wouldn't use pip install -I any time I encounter this by hand, so I can't see any reason not to make that the default.
Updated by Tim Pierce over 9 years ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|commit:59963e1ee432a51a3f7ae3d98068a541b2738879.
Updated by Tim Pierce over 9 years ago
- Subject changed from google-api-python-client fails to install to [Crunch] google-api-python-client fails to install
- Category set to Crunch
Updated by Tim Pierce over 9 years ago
Moved to state 'feedback' so we can confirm the fix once deployed.
Updated by Tom Clegg about 9 years ago
- Assigned To changed from Tim Pierce to Brett Smith
Updated by Brett Smith about 9 years ago
- Status changed from Feedback to New
- Target version changed from 2015-01-28 Sprint to 2015-02-18 sprint
qr1hi-8i9sb-o9719vud1ro9wzh successfully reproduced the issue. Unfortunately this is not fixed.
Updated by Brett Smith about 9 years ago
- Status changed from New to Resolved
The original job didn't specify an arvados_sdk_version, so changing crunch-job's install process for that would not change anything. My reproducing job did specify an arvados_sdk_version, but pip -I was still a noop because the Docker image doesn't include virtualenv.
The problem is simply that the Docker image has both the apiclient and googleapiclient modules installed. As long as that's true, this exception will be raised on import. There's nothing we can do about it on the Crunch level.
Bryan, I believe you've effectively worked around this by adding virtualenv to the image and using arvados_sdk_version, but next time you're in there, you might want to clean up by removing the old apiclient Python module (pip uninstall apiclient) in favor of googleapiclient. Let me know if you have any questions about it.