Feature #17013
closed
Release python wheels
Description
https://github.com/arvados/arvados/pull/73
Related, I noticed this code in run_upload_packages.py
# NOTE: If we ever start uploading Python 3 packages, we'll need to
# figure out some way to adapt cmd to match. It might be easiest
# to give all our setup.py files the executable bit, and run that
# directly.
# We also must run `sdist` before `upload`: `upload` uploads any
# distributions previously generated in the command. It doesn't
# know how to upload distributions already on disk. We write the
# result to a dedicated directory to avoid interfering with our
# timestamp tracking.
cmd = ['python2.7', 'setup.py']
Updated by Peter Amstutz about 4 years ago
- Target version changed from 2020-11-04 Sprint to 2020-11-18
Updated by Peter Amstutz about 4 years ago
- Target version changed from 2020-11-18 to 2020-12-02 Sprint
Updated by Ward Vandewege about 4 years ago
- Status changed from New to In Progress
# NOTE: If we ever start uploading Python 3 packages, we'll need to
# figure out some way to adapt cmd to match. It might be easiest
# to give all our setup.py files the executable bit, and run that
# directly.
# We also must run `sdist` before `upload`: `upload` uploads any
# distributions previously generated in the command. It doesn't
# know how to upload distributions already on disk. We write the
# result to a dedicated directory to avoid interfering with our
# timestamp tracking.
cmd = ['python2.7', 'setup.py']
I've fixed that to use python3.
Updated by Ward Vandewege about 4 years ago
Ready for review in commit:c44efafc9e35941d0ebf1fdc819cd848024b82ff on branch 17013-upload-wheels-to-pypi in the arvados-dev repository.
This doesn't upgrade to using 'twine' for upload, which is now apparently preferred (cf. https://packaging.python.org/guides/distributing-packages-using-setuptools/?highlight=--dist-dir#upload-your-distributions). We don't have a recent version of twine on our build server yet.
When this is merged I intend to merge PR #73 (https://github.com/arvados/arvados/pull/73) to make the wheels universal for the Arvados Python SDK. I will see about making the same change for arvados-cwl-runner, and if possible, arvados-fuse (if llfuse doesn't get in the way).
Updated by Peter Amstutz about 4 years ago
Ward Vandewege wrote:
Ready for review in commit:c44efafc9e35941d0ebf1fdc819cd848024b82ff on branch 17013-upload-wheels-to-pypi in the arvados-dev repository.
This doesn't upgrade to using 'twine' for upload, which is now apparently preferred (cf. https://packaging.python.org/guides/distributing-packages-using-setuptools/?highlight=--dist-dir#upload-your-distributions). We don't have a recent version of twine on our build server yet.
All the documentation says to use twine, so we will definitely want to do that. But if "setup.py upload" still works that's fine for now.
When this is merged I intend to merge PR #73 (https://github.com/arvados/arvados/pull/73) to make the wheels universal for the Arvados Python SDK. I will see about making the same change for arvados-cwl-runner, and if possible, arvados-fuse (if llfuse doesn't get in the way).
I don't think we actually want to make universal wheels? That means they work on both Python 2 and Python 3, and we don't support Python 2 any more. We want pure-python py3 wheels.
Updated by Ward Vandewege about 4 years ago
- % Done changed from 0 to 100
- Status changed from In Progress to Resolved
Applied in changeset arvados-dev|e1c58d5713f77dbf1adaa87fa80c2a6afef99824.
Updated by Ward Vandewege about 4 years ago
- Related to Bug #17159: [build] make build-packages-python-ruby run on a Jenkins satellite added
Updated by Ward Vandewege about 4 years ago
- Related to Bug #17160: [packaging] upgrade run_upload_packages.py to use twine for upload to PyPI added
Updated by Ward Vandewege about 4 years ago
Peter Amstutz wrote:
This doesn't upgrade to using 'twine' for upload, which is now apparently preferred (cf. https://packaging.python.org/guides/distributing-packages-using-setuptools/?highlight=--dist-dir#upload-your-distributions). We don't have a recent version of twine on our build server yet.
All the documentation says to use twine, so we will definitely want to do that. But if "setup.py upload" still works that's fine for now.
Yes, "setup.py upload" still works and it has worked for many years. twine just seems like the latest fad. I'm sure there will be another tool that will be "strongly recommended" in a year or two, while all the other ways keep working. Pfff.
We should make this CI job run in a satellite instance that is more up to date, and then we can switch to using twine. I've made a few tickets for that work (#17159 and #17160).
When this is merged I intend to merge PR #73 (https://github.com/arvados/arvados/pull/73) to make the wheels universal for the Arvados Python SDK. I will see about making the same change for arvados-cwl-runner, and if possible, arvados-fuse (if llfuse doesn't get in the way).
I don't think we actually want to make universal wheels? That means they work on both Python 2 and Python 3, and we don't support Python 2 any more. We want pure-python py3 wheels.
Fair enough. This patch builds "pure" wheels, so then we are all set.