Project

General

Profile

Actions

Story #20311

open

Update Python packages to build with PEP 517/518

Added by Brett Smith 11 months ago. Updated 4 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Deployment
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Assuming we want to continue to build with setuptools, this means converting our packages' current setup.py to setup.cfg, then adding a (boilerplate) setup.py and pyproject.toml to them.

Reasons to do this:

  • Gives us the option to build packages using other systems in the future if we need or want that for any reason.
  • In my experience, maintining setup.cfg is nicer than setup.py, since it's more declarative.
  • Some of the interfaces of setup.py are deprecated (I think we're already getting warnings about this), and this gives us a clear migration path off those too.

Background reading:


Related issues

Related to Arvados - Story #20723: Stop running setup.py in our build+test infrastructureNew

Actions
Actions #1

Updated by Brett Smith 8 months ago

  • Related to Story #20723: Stop running setup.py in our build+test infrastructure added
Actions #2

Updated by Brett Smith 5 months ago

Python 3.12 will not preinstall setuptools in new virtualenvs. This will break our current build processes in various places. Doing this ticket would be the best way to get ahead of things.

Actions #3

Updated by Brett Smith 4 months ago

Brett Smith wrote:

  • Some of the interfaces of setup.py are deprecated (I think we're already getting warnings about this), and this gives us a clear migration path off those too.

To put more color on this: all of the interfaces of setup.py are deprecated. As the top of that post says,

This does not mean that setuptools itself is deprecated, or that using setup.py to configure your package builds is going to be removed. The only thing you must stop doing is directly executing the setup.py file — instead delegate that to purpose-built or standards-based tools, preferably those that work with any build backend.

We have been seeing this deprecation start to cause issues in some of our dependencies, which makes me think this migration is becoming more urgent.

For us, the path I think we want to take is: wherever we run setup.py install, run pip instead. We've already had a couple commits along these lines: see 20432a4533136a5ab9fa52c2e2ec2d90a855ecfb and upcoming d1e529905d4820ce450dc430139cccda83fefc72.

Wherever we run setup.py build, that needs to be updated to something like python -m build using the build module. There are other alternatives, this is just the simplest one I know about. However, before we can do that, we need to migrate to using at least pyproject.toml as specified in this ticket. Migrating setup.py build calls can either be part of this ticket, or a separate follow-up.

Actions

Also available in: Atom PDF