Package Arvados for Ubuntu Bionic Beaver 18.04 LTS
It would be helpful to have arvados apt packages for ubuntu bionic, the "new" LTS release.
One motivating reason is that none of the currently supported Debian/Ubuntu releases have a slurm package with functionality new enough to support the priority ranges Arvados uses when large number of containers are run (see issue 13489).
Bionic has slurm 17.11.2 which would alleviate this issue.
With the pip3 change I can build bionic packages, but the python packages have dependency problems.
"Depends: python-ws4py (< 0.4) but 0.4.2+dfsg1-2 is to be installed"
(python-arvados-python-client depends on ws4py<0.4 because tests failed with 0.4.1 -- but ubuntu:bionic ships 0.4.2)
SDK tests seem to pass with ws4py 0.4.2.
Same problem with ruamel.yaml: bionic ships 0.15.34-1 but sdk/python requires <0.15.
python-arvados-python-client : Depends: python-ruamel.yaml (< 0.15) but 0.15.34-1 is to be installed
13823-bionic @ f8f3944e6ba1f470eedf48aa609a50c780fe4b6d
...successfully produces packages for all distros on my dev box. Haven't tried test-packages.Incidental changes (see commit comments for more details):
- switch from cure-fpm to fpm
- upgrade ws4py and ruamel.yaml python deps
#11 Updated by Ward Vandewege 6 months ago
I reviewed and made some more improvements on this branch, latest at bc97d806fa1fb15e49cefe819f00089ebc33d376
- fixed package testing for bionic
- merged latest master
From my perspective this is now ready for merging. Then:
- add the build steps to CI to build, upload and publish the packages in -dev
- do some functional testing (i.e. do the packages actually create a working version of Arvados?) on a bionic system
- update the https://doc.arvados.org/install/install-manual-prerequisites.html page to list bionic as a supported distribution
#17 Updated by Ward Vandewege 3 months ago
Tom Morris wrote:
This was tagged for the upcoming release, but I can't tell if it's done or not. Merged, but needs to be tested still?
Everything is done in this ticket, but there is a known issue with a cwltool/ruamel version incompatibility with bionic, which is being handled in #14483. Once that is merged, this should work, and we can test our 18.04 packaging.
#18 Updated by Ward Vandewege 3 months ago
Well, the ruamel issue is resolved, but there's one more dependency issue, now with python-avro:
root@1aa90ea77c25:/etc/apt/sources.list.d# apt-get install python-schema-salad Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: python-schema-salad : Depends: python-avro (< 1.8.2) but 1.8.2+dfsg-1 is to be installed E: Unable to correct problems, you have held broken packages. root@1aa90ea77c25:/etc/apt/sources.list.d# apt-cache policy python-avro python-avro: Installed: (none) Candidate: 1.8.2+dfsg-1 Version table: 1.8.2+dfsg-1 500 500 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages 1.8.1-2 500 500 http://apt.arvados.org bionic-dev/main amd64 Packages
- Assigned To changed from Tom Clegg to Ward Vandewege
- update schema-salad so it works with avro≥1.8.2
- update schema-salad to use the avro-cwl fork in python2 (avro-cwl is at 1.8.4, but it's used by schema-salad only when in python3)
- update a-c-r packaging to use virtualenv for python deps