Idea #12268
closedUpdate to standard libcloud 2.x
Added by Tom Morris almost 7 years ago. Updated about 6 years ago.
Description
We want to stop using a private fork of libcloud, but we need the regression in the current version fixed and release first.
PR: https://github.com/apache/libcloud/pull/1110
Libcloud 2.2.1 was released September 21, 2017.
Updated by Tom Morris almost 7 years ago
- Tracker changed from Bug to Idea
- Description updated (diff)
- Story points set to 0.5
Updated by Lucas Di Pentima almost 7 years ago
- Assigned To set to Lucas Di Pentima
Updated by Lucas Di Pentima almost 7 years ago
- Target version changed from 2017-10-11 Sprint to 2017-10-25 Sprint
Updated by Lucas Di Pentima almost 7 years ago
- Target version changed from 2017-10-25 Sprint to 2017-11-08 Sprint
Updated by Lucas Di Pentima almost 7 years ago
- Target version changed from 2017-11-08 Sprint to 2017-11-22 Sprint
Updated by Lucas Di Pentima almost 7 years ago
- Target version changed from 2017-11-22 Sprint to 2017-12-06 Sprint
Updated by Lucas Di Pentima over 6 years ago
- Target version changed from 2017-12-06 Sprint to 2017-12-20 Sprint
Updated by Lucas Di Pentima over 6 years ago
- Target version changed from 2017-12-20 Sprint to 2018-01-17 Sprint
Updated by Tom Morris over 6 years ago
- Target version changed from 2018-01-17 Sprint to 2018-01-31 Sprint
Updated by Lucas Di Pentima over 6 years ago
Libcloud dev version includes 4 PRs submitted by us, as described on https://github.com/apache/libcloud/blob/trunk/CHANGES.rst
Updated by Tom Morris over 6 years ago
The last of these PRs was merged Nov. 9, 2017 so any release dated later than that should be OK.
The latest release currently is still Libcloud 2.2.1, which was released September 21, 2017.
Updated by Lucas Di Pentima over 6 years ago
- Target version changed from 2018-01-31 Sprint to 2018-02-14 Sprint
Updated by Lucas Di Pentima over 6 years ago
- Status changed from New to In Progress
Updated by Lucas Di Pentima over 6 years ago
Update at 318d42f0d - branch 12268-libcloud-2-3-0
Test run: https://ci.curoverse.com/job/developer-run-tests/563/
- Made a new branch on our
libcloud
fork with 2.3.0 RC version (https://github.com/curoverse/libcloud/tree/apache-libcloud-2.2.2.dev4) - Updated
nodemanager
dependency to this version so we can do some testing. - Ran local nodemanager integration tests: passed OK, now running all the tests on Jenkins before merging.
Updated by Lucas Di Pentima over 6 years ago
- Target version changed from 2018-02-14 Sprint to 2018-02-28 Sprint
Updated by Lucas Di Pentima over 6 years ago
Latest release still is 2.2.1 as of Feb 28th.
Updated by Lucas Di Pentima over 6 years ago
- Target version changed from 2018-02-28 Sprint to 2018-03-14 Sprint
Updated by Lucas Di Pentima over 6 years ago
Libcloud 2.3.0 is released. The final commit wasn't exactly the one that we used as 2.2.2.dev4 but close, here's the diff:
Updated by Lucas Di Pentima over 6 years ago
Updates at 8e2a634e2 - branch 12268-libcloud230-official
Test run: https://ci.curoverse.com/job/developer-run-tests/629/
Updates nodemanager's setup.py
to request libcloud>=2.3
, also updated run-tests.sh
so it gets the package from PyPI when it doesn't exist as a fork.
There's a question pending to be answered about version requirements: should be keep asking >=2.3 or something more restrictive?
Updated by Tom Clegg over 6 years ago
Do we still need this line in run-tests.sh? It seems like it prefers installing the curoverse version 2.3.0, and only falls back on pypi if that fails. Shouldn't we just go straight to pypi?
|| pip install --pre --ignore-installed https://github.com/curoverse/libcloud/archive/apache-libcloud-$LIBCLOUD_PIN.zip >/dev/null \
rest LGTM, thanks.
Updated by Lucas Di Pentima over 6 years ago
Update at 69ae9c5a8d7e2df93ece51afd4eb471c745eb84c
Test run: https://ci.curoverse.com/job/developer-run-tests/630/
Removed libcloud installation attempt from our forked version.
Updated by Lucas Di Pentima over 6 years ago
- Status changed from In Progress to Resolved
- % Done changed from 50 to 100
Applied in changeset arvados|07a09069c579e51c33c6fb267b2e510cc8f98ea8.
Updated by Ward Vandewege over 6 years ago
- Status changed from Resolved to In Progress
Hmm all package building is now broken because the package testing fails, cf. https://ci.curoverse.com/job/build-packages-debian9/303/console for example:
02:04:14 START: arvados-node-manager test on arvados/package-test:debian9 02:04:17 W: The repository 'file:/arvados/packages/debian9 Release' is not signed. 02:04:18 Reading package lists... 02:04:18 Building dependency tree... 02:04:18 Reading state information... 02:04:18 Some packages could not be installed. This may mean that you have 02:04:18 requested an impossible situation or if you are using the unstable 02:04:18 distribution that some required packages have not yet been created 02:04:18 or been moved out of Incoming. 02:04:18 The following information may help to resolve the situation: 02:04:18 02:04:18 The following packages have unmet dependencies: 02:04:18 arvados-node-manager : Depends: python-apache-libcloud (>= 2.3) but it is not going to be installed 02:04:18 E: Unable to correct problems, you have held broken packages. 02:04:19 ERROR: arvados-node-manager test on arvados/package-test:debian9 failed with exit status 100 02:04:19
Updated by Lucas Di Pentima over 6 years ago
Branch 12268-build-fix
fixes the package build issue by adding apache-libcloud
to build.list
file, and removing the libcloud fork building code.
Updated by Nico César over 6 years ago
review at c6e1f628cc07731cd5fdee991fbd49564dab44b7
I see that this got eliminated:
# Forked libcloud if test_package_presence "$PYTHON2_PKG_PREFIX"-apache-libcloud "$LIBCLOUD_PIN" python 2 then LIBCLOUD_DIR=$(mktemp -d) ( cd $LIBCLOUD_DIR git clone $DASHQ_UNLESS_DEBUG https://github.com/curoverse/libcloud.git . git checkout $DASHQ_UNLESS_DEBUG apache-libcloud-$LIBCLOUD_PIN # libcloud is absurdly noisy without -q, so force -q here OLD_DASHQ_UNLESS_DEBUG=$DASHQ_UNLESS_DEBUG DASHQ_UNLESS_DEBUG=-q handle_python_package DASHQ_UNLESS_DEBUG=$OLD_DASHQ_UNLESS_DEBUG ) fpm_build $LIBCLOUD_DIR "$PYTHON2_PKG_PREFIX"-apache-libcloud "" python "" --iteration 2 rm -rf $LIBCLOUD_DIR fi
which is good ... but I have the strong feeling that we'll be forking again sometime in the future
can you add the following comment instead:
## if libcloud becomes our own fork see ## https://dev.arvados.org/issues/12268#note-27
besides that is ready to merge.
Updated by Lucas Di Pentima over 6 years ago
- Status changed from In Progress to Resolved