Arvados

Peter Amstutz wrote:

Debian 10, 11 and 12

Debian 10 has 3.7 and trying to get a more recent one is a huge task.

11 has 3.9 and 12 has 3.11, those are fine.

Ubuntu 18.04, 20.04, 22.04

18.04 itself is out of LTS so I vote we drop it anyway for that reason.

20.04 has 3.8 and 22.04 has 3.10, those are fine.

Rocky 8.6

Rocky 8.8, at least, has a python39 stream available. I'm not sure how big a pain it is to activate and use but at least the option is there.

The fact that our packages are built on Rocky 8.6 is not deeply considered, it was just the oldest version of Rocky that had a Docker image at the time we added it, and it had enough support to work.

Brett Smith wrote in #note-4:

20.04 has 3.8 and 22.04 has 3.10, those are fine.

I got a little curious and looked ahead a bit, Python 3.8 has support until October 2024. So I'd guess we'll be looking to drop support for Python 3.8 around the same time Ubuntu drops support for all of 20.04.

Brett Smith wrote in #note-4:

Rocky 8.8, at least, has a python39 stream available. I'm not sure how big a pain it is to activate and use but at least the option is there.

https://www.redhat.com/en/blog/introduction-appstreams-and-modules-red-hat-enterprise-linux

How is that different from Red Hat Software Collections you might ask? Software Collections keeps multiple versions on one system, putting each package into separate namespaced paths. Modularity uses the standard RPM packaging, so paths are where you expect them to be. Keep in mind, unlike Software Collections, modules do not allow running multiple versions at the same time. Only one stream can be installed and used at a time.

So, it seems easy enough to use: we just add stream specifiers to some of the packages installed in the Dockerfile, and test those get picked up as dependencies. But note this means that if we go this route, any administrator who uses our packages will be required to have that version of Python installed across the entire system. Stuff that's included with the distro should cause no problems, but if they have any other third-party applications that depend on a different Python version stream, there'll be an unresolvable conflict between that and the Arvados package.

I don't know how likely this is. We could ask our users.

Brett Smith wrote in #note-7:

I don't know how likely this is. We could ask our users.

By coincidence, our Red Hat users are not using Docker cleaner, which means they have no need for our Python packages on the server side. They only need client side tools.

So one option we have is to start doing #20838, at least for distros that do not include Python>=3.8. The interim rule could be: if your distro does not include a new-enough Python, then we do not build Python packages for it. You can still install the software by getting a newer Python any way you want (build from source/appstream/backport package/whatever) and then installing inside a virtualenv. Most of our client software already has instructions for how to do the latter part; we can review and address any gaps as part of this.

We are dropping support for Debian 10 since it ships with Python 3.7. I have removed build-packages-debian10 from build-packages-multijob.

We are going to need to update test-provision-debian10, I'll make a new ticket for that.