Bug #9242

[Deployment] python-six backport breaks other users

Added by Brett Smith over 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Brett Smith
Category:
Deployment
Target version:
Start date:
05/20/2016
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-

Description

The arvados supplied python-six deb (1.10.0-1) is a newer version than the stock ubuntu 14.04 version (1.5.2-1ubuntu1), so replaces it. The version supplied by ubuntu installs to /usr/lib/python2.7/dist-packages/ as expected, but the arvados version installs to /usr/local/lib/python2.7/dist-packages.

The stock ubuntu python-scipy package contains a symbolic link as follows:
/usr/lib/python2.7/dist-packages/scipy/lib/six.py -> ../../six.py

With the stock python-six installed this is fine, but of course with the arvados version this symlink is broken.

Current thinking is that the best way to fix this is to have our backport install its files to /usr/lib instead of /usr/local/lib. If we want it to act as a drop-in replacement package, it should maintain as much compatibility as possible, including having files in the original locations.

pkgtest.log.xz (34.4 KB) pkgtest.log.xz Brett Smith, 05/20/2016 07:52 PM

Subtasks

Task #9247: Review 9242-python-backport-prefix-wipResolvedWard Vandewege

Associated revisions

Revision ae72b172
Added by Brett Smith about 3 years ago

Merge branch '9242-python-backport-prefix-wip'

Closes #9242, #9247.

Revision 3aae316c (diff)
Added by Brett Smith about 3 years ago

9242: Update Python module paths for CentOS 6.

I am more sure that this is correct, based on multiple data points
from Python 2 and 3 packages across CentOS 6 and 7.
This might be a change that's fallout from
44ceaa474a330f12dd9e00115af107d7258044f2.
Refs #9242.

Revision 76549a60 (diff)
Added by Brett Smith about 3 years ago

9242: Restore newer backported versions of Python packages.

I accidentally reverted this in 758d39f.
Refs #9242.

History

#1 Updated by Brett Smith over 3 years ago

  • Status changed from New to In Progress
  • Assigned To set to Brett Smith
  • Target version set to 2016-05-25 sprint

#2 Updated by Brett Smith over 3 years ago

Brett Smith wrote:

Current thinking is that the best way to fix this is to have our backport install its files to /usr/lib instead of /usr/local/lib. If we want it to act as a drop-in replacement package, it should maintain as much compatibility as possible, including having files in the original locations.

9242-python-backport-prefix-wip is up for review to implement this. I have built and tested the packages on all targets. See the attached test log.

#3 Updated by Brett Smith over 3 years ago

  • Target version changed from 2016-05-25 sprint to 2016-06-08 sprint

#4 Updated by Ward Vandewege about 3 years ago

reviewing 9242-python-backport-prefix-wip: LGTM. Thanks for the various cleanups.

#5 Updated by Brett Smith about 3 years ago

Ward Vandewege wrote:

reviewing 9242-python-backport-prefix-wip: LGTM. Thanks for the various cleanups.

Dealing with a merge conflict, I realized I had a bug: I took the --iteration switch out of a few lines, not realizing that they were calling fpm directly instead of fpm_build.

I have rebased the branch on master, and updated those lines to use fpm_build, so they'll continue to benefit from smarts that we add there. Now at 970fab5. The only lines that are changed relative to the previous review is the hunk referring to schema_salad et al.

#6 Updated by Ward Vandewege about 3 years ago

Brett Smith wrote:

Ward Vandewege wrote:

reviewing 9242-python-backport-prefix-wip: LGTM. Thanks for teh various cleanups.

Dealing with a merge conflict, I realized I had a bug: I took the --iteration switch out of a few lines, not realizing that they were calling fpm directly instead of fpm_build.

Oh! I thought that was on purpose, since they were superfluous (--iteration 1).

I have rebased the branch on master, and updated those lines to use fpm_build, so they'll continue to benefit from smarts that we add there. Now at 970fab5. The only lines that are changed relative to the previous review is the hunk referring to schema_salad et al.

Very nice. Great improvement. You can drop the --verbose --log info from the rdflib-jsonld build line, there's no reason for that. Of course this now bumps the iteration to 2 for the schema_salad, ruamel.yaml, cwltool and rdflib-jsonld builds, for no real reason. Maybe add --iteration 1 to those lines?

Otherwise, LGTM!

#7 Updated by Brett Smith about 3 years ago

Ward Vandewege wrote:

Brett Smith wrote:

Dealing with a merge conflict, I realized I had a bug: I took the --iteration switch out of a few lines, not realizing that they were calling fpm directly instead of fpm_build.

Oh! I thought that was on purpose, since they were superfluous (--iteration 1).

That's not superfluous: if you run fpm without any --iteration, you get a package with no iteration, which is different than a package with iteration 1. But the reason I took them out is because my intent was to let fpm_build set the iteration. Like I said, I thought they were fpm_build lines, and now they are.

I have rebased the branch on master, and updated those lines to use fpm_build, so they'll continue to benefit from smarts that we add there. Now at 970fab5. The only lines that are changed relative to the previous review is the hunk referring to schema_salad et al.

Very nice. Great improvement. You can drop the --verbose --log info from the rdflib-jsonld build line, there's no reason for that.

Done in 5520e6c.

Of course this now bumps the iteration to 2 for the schema_salad, ruamel.yaml, cwltool and rdflib-jsonld builds, for no real reason.

There is a real reason: all the files have moved around inside the package. (This is the functional reason to switch to fpm_build: so the packages use distro-like paths instead of /usr/local.) Since there's a non-functional package change, bumping the iteration is the right thing to do.

Thanks.

#8 Updated by Ward Vandewege about 3 years ago

Brett Smith wrote:

Ward Vandewege wrote:

Brett Smith wrote:

Dealing with a merge conflict, I realized I had a bug: I took the --iteration switch out of a few lines, not realizing that they were calling fpm directly instead of fpm_build.

Oh! I thought that was on purpose, since they were superfluous (--iteration 1).

That's not superfluous: if you run fpm without any --iteration, you get a package with no iteration, which is different than a package with iteration 1. But the reason I took them out is because my intent was to let fpm_build set the iteration. Like I said, I thought they were fpm_build lines, and now they are.

I have rebased the branch on master, and updated those lines to use fpm_build, so they'll continue to benefit from smarts that we add there. Now at 970fab5. The only lines that are changed relative to the previous review is the hunk referring to schema_salad et al.

Very nice. Great improvement. You can drop the --verbose --log info from the rdflib-jsonld build line, there's no reason for that.

Done in 5520e6c.

Of course this now bumps the iteration to 2 for the schema_salad, ruamel.yaml, cwltool and rdflib-jsonld builds, for no real reason.

There is a real reason: all the files have moved around inside the package. (This is the functional reason to switch to fpm_build: so the packages use distro-like paths instead of /usr/local.) Since there's a non-functional package change, bumping the iteration is the right thing to do.

Thanks.

OK makes sense. LGTM.

#9 Updated by Brett Smith about 3 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:ae72b172c8bf8a52358a89f8a8d744ec5bf2d993.

Also available in: Atom PDF