Update to standard libcloud 2.x
We want to stop using a private fork of libcloud, but we need the regression in the current version fixed and release first.
Libcloud 2.2.1 was released September 21, 2017.
#11 Updated by Lucas Di Pentima almost 2 years ago
Libcloud dev version includes 4 PRs submitted by us, as described on https://github.com/apache/libcloud/blob/trunk/CHANGES.rst
#15 Updated by Lucas Di Pentima over 1 year ago
Update at 318d42f0d - branch
Test run: https://ci.curoverse.com/job/developer-run-tests/563/
- Made a new branch on our
libcloudfork with 2.3.0 RC version (https://github.com/curoverse/libcloud/tree/apache-libcloud-2.2.2.dev4)
nodemanagerdependency to this version so we can do some testing.
- Ran local nodemanager integration tests: passed OK, now running all the tests on Jenkins before merging.
#20 Updated by Lucas Di Pentima over 1 year ago
Libcloud 2.3.0 is released. The final commit wasn't exactly the one that we used as 2.2.2.dev4 but close, here's the diff:
#21 Updated by Lucas Di Pentima over 1 year ago
Updates at 8e2a634e2 - branch
Test run: https://ci.curoverse.com/job/developer-run-tests/629/
setup.py to request
libcloud>=2.3, also updated
run-tests.sh so it gets the package from PyPI when it doesn't exist as a fork.
There's a question pending to be answered about version requirements: should be keep asking >=2.3 or something more restrictive?
#22 Updated by Tom Clegg over 1 year ago
Do we still need this line in run-tests.sh? It seems like it prefers installing the curoverse version 2.3.0, and only falls back on pypi if that fails. Shouldn't we just go straight to pypi?
|| pip install --pre --ignore-installed https://github.com/curoverse/libcloud/archive/apache-libcloud-$LIBCLOUD_PIN.zip >/dev/null \
rest LGTM, thanks.
#23 Updated by Lucas Di Pentima over 1 year ago
Removed libcloud installation attempt from our forked version.
#25 Updated by Ward Vandewege over 1 year ago
- Status changed from Resolved to In Progress
Hmm all package building is now broken because the package testing fails, cf. https://ci.curoverse.com/job/build-packages-debian9/303/console for example:
02:04:14 START: arvados-node-manager test on arvados/package-test:debian9 02:04:17 W: The repository 'file:/arvados/packages/debian9 Release' is not signed. 02:04:18 Reading package lists... 02:04:18 Building dependency tree... 02:04:18 Reading state information... 02:04:18 Some packages could not be installed. This may mean that you have 02:04:18 requested an impossible situation or if you are using the unstable 02:04:18 distribution that some required packages have not yet been created 02:04:18 or been moved out of Incoming. 02:04:18 The following information may help to resolve the situation: 02:04:18 02:04:18 The following packages have unmet dependencies: 02:04:18 arvados-node-manager : Depends: python-apache-libcloud (>= 2.3) but it is not going to be installed 02:04:18 E: Unable to correct problems, you have held broken packages. 02:04:19 ERROR: arvados-node-manager test on arvados/package-test:debian9 failed with exit status 100 02:04:19
#27 Updated by Nico César over 1 year ago
review at c6e1f628cc07731cd5fdee991fbd49564dab44b7
I see that this got eliminated:
# Forked libcloud if test_package_presence "$PYTHON2_PKG_PREFIX"-apache-libcloud "$LIBCLOUD_PIN" python 2 then LIBCLOUD_DIR=$(mktemp -d) ( cd $LIBCLOUD_DIR git clone $DASHQ_UNLESS_DEBUG https://github.com/curoverse/libcloud.git . git checkout $DASHQ_UNLESS_DEBUG apache-libcloud-$LIBCLOUD_PIN # libcloud is absurdly noisy without -q, so force -q here OLD_DASHQ_UNLESS_DEBUG=$DASHQ_UNLESS_DEBUG DASHQ_UNLESS_DEBUG=-q handle_python_package DASHQ_UNLESS_DEBUG=$OLD_DASHQ_UNLESS_DEBUG ) fpm_build $LIBCLOUD_DIR "$PYTHON2_PKG_PREFIX"-apache-libcloud "" python "" --iteration 2 rm -rf $LIBCLOUD_DIR fi
which is good ... but I have the strong feeling that we'll be forking again sometime in the future
can you add the following comment instead:
## if libcloud becomes our own fork see ## https://dev.arvados.org/issues/12268#note-27
besides that is ready to merge.