Project

General

Profile

Actions

Bug #21087

closed

Python 3.7 deprecation

Added by Peter Amstutz 8 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Story points:
-

Description

Python 3.7 is no longer supported and is being dropped from upstream packages that we depend on.

For our supported distributions, we need to either:

  1. Ensure that we are installing and depend on a Python > 3.7, if available
  2. Drop those distributions

Distributions to check include:

Debian 10, 11 and 12

Ubuntu 18.04, 20.04, 22.04

Rocky 8.6

(we already plan to drop CentOS 7)


Related issues

Related to Arvados - Support #20838: Discontinue OS packages of client tools in favor of pypi/rubygems onlyNewActions
Related to Arvados - Bug #21273: Package builds on Py3.6 distros fail - cannot install setuptools>=60 but we're requesting >=66ResolvedBrett SmithActions
Related to Arvados - Bug #21367: build-dev-docker-jobs-image.sh won't buildResolvedBrett SmithActions
Related to Arvados - Feature #21388: Update list of supported distributions everywhereResolvedBrett SmithActions
Related to Arvados - Idea #21406: Declare python_requires~=3.8 for all our Python packagesResolvedBrett SmithActions
Actions #1

Updated by Peter Amstutz 8 months ago

  • Status changed from New to In Progress
Actions #2

Updated by Peter Amstutz 8 months ago

  • Status changed from In Progress to New
  • Description updated (diff)
Actions #3

Updated by Peter Amstutz 8 months ago

  • Description updated (diff)
Actions #4

Updated by Brett Smith 8 months ago

Peter Amstutz wrote:

Debian 10, 11 and 12

Debian 10 has 3.7 and trying to get a more recent one is a huge task.

11 has 3.9 and 12 has 3.11, those are fine.

Ubuntu 18.04, 20.04, 22.04

18.04 itself is out of LTS so I vote we drop it anyway for that reason.

20.04 has 3.8 and 22.04 has 3.10, those are fine.

Rocky 8.6

Rocky 8.8, at least, has a python39 stream available. I'm not sure how big a pain it is to activate and use but at least the option is there.

The fact that our packages are built on Rocky 8.6 is not deeply considered, it was just the oldest version of Rocky that had a Docker image at the time we added it, and it had enough support to work.

Actions #5

Updated by Brett Smith 8 months ago

Brett Smith wrote in #note-4:

20.04 has 3.8 and 22.04 has 3.10, those are fine.

I got a little curious and looked ahead a bit, Python 3.8 has support until October 2024. So I'd guess we'll be looking to drop support for Python 3.8 around the same time Ubuntu drops support for all of 20.04.

Actions #6

Updated by Peter Amstutz 8 months ago

  • Target version changed from Development 2023-11-08 sprint to Development 2023-11-29 sprint
Actions #7

Updated by Brett Smith 7 months ago

Brett Smith wrote in #note-4:

Rocky 8.8, at least, has a python39 stream available. I'm not sure how big a pain it is to activate and use but at least the option is there.

https://www.redhat.com/en/blog/introduction-appstreams-and-modules-red-hat-enterprise-linux

How is that different from Red Hat Software Collections you might ask? Software Collections keeps multiple versions on one system, putting each package into separate namespaced paths. Modularity uses the standard RPM packaging, so paths are where you expect them to be. Keep in mind, unlike Software Collections, modules do not allow running multiple versions at the same time. Only one stream can be installed and used at a time.

So, it seems easy enough to use: we just add stream specifiers to some of the packages installed in the Dockerfile, and test those get picked up as dependencies. But note this means that if we go this route, any administrator who uses our packages will be required to have that version of Python installed across the entire system. Stuff that's included with the distro should cause no problems, but if they have any other third-party applications that depend on a different Python version stream, there'll be an unresolvable conflict between that and the Arvados package.

I don't know how likely this is. We could ask our users.

Actions #8

Updated by Peter Amstutz 7 months ago

  • Target version changed from Development 2023-11-29 sprint to Development 2024-01-03 sprint
Actions #9

Updated by Brett Smith 7 months ago

Brett Smith wrote in #note-7:

I don't know how likely this is. We could ask our users.

By coincidence, our Red Hat users are not using Docker cleaner, which means they have no need for our Python packages on the server side. They only need client side tools.

So one option we have is to start doing #20838, at least for distros that do not include Python>=3.8. The interim rule could be: if your distro does not include a new-enough Python, then we do not build Python packages for it. You can still install the software by getting a newer Python any way you want (build from source/appstream/backport package/whatever) and then installing inside a virtualenv. Most of our client software already has instructions for how to do the latter part; we can review and address any gaps as part of this.

Actions #10

Updated by Brett Smith 7 months ago

  • Related to Support #20838: Discontinue OS packages of client tools in favor of pypi/rubygems only added
Actions #11

Updated by Peter Amstutz 7 months ago

  • Target version changed from Development 2024-01-03 sprint to Development 2024-01-17 sprint
Actions #12

Updated by Brett Smith 6 months ago

  • Related to Bug #21273: Package builds on Py3.6 distros fail - cannot install setuptools>=60 but we're requesting >=66 added
Actions #13

Updated by Brett Smith 5 months ago

  • Assigned To set to Brett Smith
Actions #14

Updated by Brett Smith 5 months ago

We are dropping support for Debian 10 since it ships with Python 3.7. I have removed build-packages-debian10 from build-packages-multijob.

We are going to need to update test-provision-debian10, I'll make a new ticket for that.

Actions #15

Updated by Brett Smith 5 months ago

  • Related to Bug #21367: build-dev-docker-jobs-image.sh won't build added
Actions #16

Updated by Peter Amstutz 5 months ago

  • Target version changed from Development 2024-01-17 sprint to Development 2024-01-31 sprint
Actions #17

Updated by Peter Amstutz 5 months ago

  • Target version changed from Development 2024-01-31 sprint to Development 2024-01-17 sprint
Actions #18

Updated by Peter Amstutz 5 months ago

  • Status changed from New to In Progress
Actions #19

Updated by Brett Smith 5 months ago

  • Related to Feature #21388: Update list of supported distributions everywhere added
Actions #20

Updated by Peter Amstutz 5 months ago

  • Status changed from In Progress to Resolved
Actions #21

Updated by Brett Smith 5 months ago

  • Related to Idea #21406: Declare python_requires~=3.8 for all our Python packages added
Actions

Also available in: Atom PDF