Feature #17013
closed
Added by Peter Amstutz over 3 years ago.
Updated over 3 years ago.
Description
https://github.com/arvados/arvados/pull/73
Related, I noticed this code in run_upload_packages.py
# NOTE: If we ever start uploading Python 3 packages, we'll need to
# figure out some way to adapt cmd to match. It might be easiest
# to give all our setup.py files the executable bit, and run that
# directly.
# We also must run `sdist` before `upload`: `upload` uploads any
# distributions previously generated in the command. It doesn't
# know how to upload distributions already on disk. We write the
# result to a dedicated directory to avoid interfering with our
# timestamp tracking.
cmd = ['python2.7', 'setup.py']
- Description updated (diff)
- Description updated (diff)
- Target version changed from 2020-11-04 Sprint to 2020-11-18
- Target version changed from 2020-11-18 to 2020-12-02 Sprint
- Assigned To set to Ward Vandewege
- Status changed from New to In Progress
# NOTE: If we ever start uploading Python 3 packages, we'll need to
# figure out some way to adapt cmd to match. It might be easiest
# to give all our setup.py files the executable bit, and run that
# directly.
# We also must run `sdist` before `upload`: `upload` uploads any
# distributions previously generated in the command. It doesn't
# know how to upload distributions already on disk. We write the
# result to a dedicated directory to avoid interfering with our
# timestamp tracking.
cmd = ['python2.7', 'setup.py']
I've fixed that to use python3.
Ward Vandewege wrote:
Ready for review in commit:c44efafc9e35941d0ebf1fdc819cd848024b82ff on branch 17013-upload-wheels-to-pypi in the arvados-dev repository.
This doesn't upgrade to using 'twine' for upload, which is now apparently preferred (cf. https://packaging.python.org/guides/distributing-packages-using-setuptools/?highlight=--dist-dir#upload-your-distributions). We don't have a recent version of twine on our build server yet.
All the documentation says to use twine, so we will definitely want to do that. But if "setup.py upload" still works that's fine for now.
When this is merged I intend to merge PR #73 (https://github.com/arvados/arvados/pull/73) to make the wheels universal for the Arvados Python SDK. I will see about making the same change for arvados-cwl-runner, and if possible, arvados-fuse (if llfuse doesn't get in the way).
I don't think we actually want to make universal wheels? That means they work on both Python 2 and Python 3, and we don't support Python 2 any more. We want pure-python py3 wheels.
- % Done changed from 0 to 100
- Status changed from In Progress to Resolved
- Related to Bug #17159: [build] make build-packages-python-ruby run on a Jenkins satellite added
- Related to Bug #17160: [packaging] upgrade run_upload_packages.py to use twine for upload to PyPI added
Peter Amstutz wrote:
This doesn't upgrade to using 'twine' for upload, which is now apparently preferred (cf. https://packaging.python.org/guides/distributing-packages-using-setuptools/?highlight=--dist-dir#upload-your-distributions). We don't have a recent version of twine on our build server yet.
All the documentation says to use twine, so we will definitely want to do that. But if "setup.py upload" still works that's fine for now.
Yes, "setup.py upload" still works and it has worked for many years. twine just seems like the latest fad. I'm sure there will be another tool that will be "strongly recommended" in a year or two, while all the other ways keep working. Pfff.
We should make this CI job run in a satellite instance that is more up to date, and then we can switch to using twine. I've made a few tickets for that work (#17159 and #17160).
When this is merged I intend to merge PR #73 (https://github.com/arvados/arvados/pull/73) to make the wheels universal for the Arvados Python SDK. I will see about making the same change for arvados-cwl-runner, and if possible, arvados-fuse (if llfuse doesn't get in the way).
I don't think we actually want to make universal wheels? That means they work on both Python 2 and Python 3, and we don't support Python 2 any more. We want pure-python py3 wheels.
Fair enough. This patch builds "pure" wheels, so then we are all set.
Also available in: Atom
PDF
Updated by 
Updated by