Bug #16434
closed
python3-arvados-cwl-runner package broken on Debian 10 and Ubuntu 18.04
Description
This is version 2.0.2 on a clean buster install:
21:27:37 + cwl-runner hasher-workflow.cwl hasher-workflow-job.yml 21:27:38 Traceback (most recent call last): 21:27:38 File "/usr/bin/cwl-runner", line 8, in <module> 21:27:38 from arvados_cwl import main 21:27:38 File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/arvados_cwl/__init__.py", line 58, in <module> 21:27:38 import cwltool.main 21:27:38 File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/main.py", line 35, in <module> 21:27:38 from . import command_line_tool, workflow 21:27:38 File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/command_line_tool.py", line 38, in <module> 21:27:38 from .docker import DockerCommandLineJob 21:27:38 File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/cwltool/docker.py", line 4, in <module> 21:27:38 from distutils import spawn 21:27:38 File "/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/distutils/__init__.py", line 44, in <module> 21:27:38 from distutils import dist, sysconfig # isort:skip 21:27:38 ImportError: cannot import name 'dist' from 'distutils' (/usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/distutils/__init__.py) 21:27:38 Build step 'Execute shell' marked build as failure
The python2 version works. Our other distro packages may also be affected. We need to add a test for this in the packaging tests (just add a simple execution of cwl-runner with a fake API host and token).
Updated by Ward Vandewege over 4 years ago
- Target version changed from To Be Groomed to 2020-05-20 Sprint
- Assigned To set to Ward Vandewege
- Status changed from New to In Progress
More information:
- this affects master as well as the 2.0 branch
- this affects Debian 10 (python 3.7) and Ubuntu 18.04 (python 3.6)
- CentOS 7 (python 3.6), Debian 9 (python 3.5) and Ubuntu 16.04 (python 3.5) are not affected
What's happening is that /usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/distutils/__init__.py goes out of its way to load the system distutils to get to stdlib. On Debian 10, the (partial) system python3 distutils comes from the libpython3.7-stdlib package, which does not include dist.py. Boom.
It's worth noting that it works fine on Debian 9 and Ubuntu 1604, where the (partial) system python3 distutils comes from the libpython3.5-stdlib package, which includes dist.py.
The workaround is easy: install the python3-distutils systemwide to provide the necessary dist.py. I added it as a dependency on our python3-arvados-cwl-runner packages for Debian 10+ and Ubuntu 18.04+.
Updated by Anonymous over 4 years ago
- % Done changed from 0 to 100
- Status changed from In Progress to Resolved
Applied in changeset arvados|595e6ca982543526862211dc132b46f949129447.
Updated by Ward Vandewege over 4 years ago
- Subject changed from python3-arvados-cwl-runner package broken on buster to python3-arvados-cwl-runner package broken on Debian 10 and Ubuntu 18.04
Updated by Javier Bértoli over 4 years ago
- Related to Bug #16611: arvados-docker-cleaner package broken on Debian 10 and Ubuntu 18.04 added
Updated by Ward Vandewege over 4 years ago
The fix on the 2.0-dev branch (in release 2.0.3 and 2.0.4) was botched, but the fix in master is correct. I pushed a fix in 9b6a214f4feb90845640bce53efa34d544c701e9 on the 2.0-dev branch, but we have no plans for a 2.0.5 at the moment. Release 2.1 will have the proper fix.