Support #20838
open
Discontinue OS packages of client tools in favor of pypi/rubygems only
Description
End users generally install from PyPi or Rubygems instead of using OS packages for client tools. This is because the OS packages have more friction: they require setting up the Arvados package repository and being root in order to install.
Meanwhile, building client packages for older OS versions with older Python/Ruby interpreters is a hassle. For example, a Python dependency drops support for an out-of-support Python (e.g. 3.6) then we have to pin an older version, fork the package, or resolve it some other way.
This is less of a problem for services -- we don't have any services that use Python, and our installer installs a specific Ruby version using rvm.
Specifically, this includes
- python3-arvados-python-client
- python3-arvados-cwl-runner
- python3-arvados-fuse
- python3-crunchstat-summary
- python3-arvados-user-activity (I don't think this one is on PyPi yet)
- python3-cwltest (IDK why we are packaging this in the first place)
Tasks:
- Announce that the client OS packages will no longer be published
- Remove them from the build scripts
- Update the documentation to only provide instructions to install from PyPi
Related issues
Updated by Peter Amstutz 9 months ago
- Related to Idea #20727: Publish standalone binaries for arvados-client, other Go client tools added
Updated by Peter Amstutz 9 months ago
- Blocked by Support #20875: Deprecate OS packages for client tools added
Updated by Brett Smith 9 months ago
What is the plan for installing shell nodes after this ticket is done? If these tools get installed via pip/gem there now too, how do the tools get into users' $PATH?
What about arvados-docker-cleaner? That's a Python service, installed on compute nodes.
Updated by Brett Smith 6 months ago
- Related to Bug #21087: Python 3.7 deprecation added
Updated by Brett Smith 5 months ago
- Related to Idea #21263: Delist or document deprecated PyPI packages added
Updated by Peter Amstutz 5 months ago
I want to reconsider this. I think there's still value in having OS packages even when most users will prefer the PyPi route. If there is a distribution we want to support which lacks a compatible Python version, we could look in to using PyInstaller (#20893) to build self-contained packages that don't rely on system Python at all.
Updated by Brett Smith 5 months ago
Peter Amstutz wrote in #note-10:
If there is a distribution we want to support which lacks a compatible Python version, we could look in to using PyInstaller (#20893) to build self-contained packages that don't rely on system Python at all.
Given our plan to support latest and previous Debian stable, and latest and previous Ubuntu LTS, the only distro we want to support that ships with too-old Python is the RHEL 8 variants. And by far the easiest way to build packages for it would be to build against the python39 appstream included with the 8.8 release.
I do think we should at least prioritize PyPI installation in the sense of making it the first documented way to install, and generally recommended. I think most users will prefer that both because it's what they're allowed to use and because they're familiar with the tools. I think we should consider the packages to be more administrator-focused, and have our documentation reflect that.
Updated by Brett Smith 4 months ago
Brett Smith wrote in #note-11:
Given our plan to support latest and previous Debian stable, and latest and previous Ubuntu LTS, the only distro we want to support that ships with too-old Python is the RHEL 8 variants. And by far the easiest way to build packages for it would be to build against the python39 appstream included with the 8.8 release.
Agreed at grooming meeting we will do this. I think #21273 is the best fit for this much-narrower-in-scope ticket.
After that, all that remains of this ticket is potential documentation changes to prioritize PyPI installs over distro ones:
I do think we should at least prioritize PyPI installation in the sense of making it the first documented way to install, and generally recommended. I think most users will prefer that both because it's what they're allowed to use and because they're familiar with the tools. I think we should consider the packages to be more administrator-focused, and have our documentation reflect that.