Project

General

Profile

Actions

Idea #8999

closed

[node manager] Upgrade to libcloud 2.0

Added by Peter Amstutz over 8 years ago. Updated about 7 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
Due date:
Story points:
2.0

Description

The pending libcloud 2.0.0 release has all our work relating to Azure ARM (including cloud environments) merged. We should test to see if node manager works with libcloud 2.0.0 so we can depend the upstream packaged version instead of our own fork.


Related issues

Related to Arvados - Bug #12055: [node manager] ec2 set tags on createResolvedLucas Di Pentima08/16/2017Actions
Blocked by Arvados - Idea #11545: Create a CWL stress test for node managerResolvedPeter Amstutz04/25/2017Actions
Actions #1

Updated by Brett Smith over 8 years ago

  • Target version set to Arvados Future Sprints
Actions #2

Updated by Tom Morris almost 8 years ago

Azure ARM support was introduced in libcloud 1.4 which was released at the end of November 2016. We should probably let it settle for a bit to make sure there are no issues with the release before contemplating updating.

We are currently using a 0.18 derivative from summer 2015.

There is a small set of upgrade notes available at http://libcloud.readthedocs.io/en/latest/upgrade_notes.html

Libcloud 1.0.0

  • Per-region provider constants and related driver classes which have been deprecated in Libcloud 0.14.0 have now been fully removed.

    Those provider drivers have moved to the single provider constant + region constructor argument in Libcloud 0.14.0.

Libcloud 0.20.0

  • New optional ttl argument has been added to libcloud.dns.base.Record class constructor before the existing extra argument.

    If you have previously manually instantiated this class and didn’t use keyword arguments, you need to update your code to correctly pass arguments to the constructor (you are encouraged to use keyword arguments to avoid such issues in the future).

  • All NodeState, StorageVolumeState, VolumeSnapshotState and Provider attributes are now strings instead of integers.

    If you are using the tostring and fromstring methods of NodeState, you are fine. If you are using NodeState.RUNNING and the like, you are also fine.

    However, if you have previously depended on these being integers, you need to update your code to depend on strings. You should consider starting using the tostring and fromstring methods as the output of these functions will not change in future versions, while the implementation might.

Libcloud 0.19.0

  • The base signature of NodeDriver.create_volume has changed. The snapshot argument is now expected to be a VolumeSnapshot instead of a string. The older signature was never correct for built-in drivers, but custom drivers may break. (GCE accepted strings, names or None and still does. Other drivers did not implement creating volumes from snapshots at all until now.)
  • VolumeSnapshots now have a created attribute that is a datetime field showing the creation datetime of the snapshot. The field in VolumeSnapshot.extra containing the original string is maintained, so this is a backwards-compatible change.
  • The OpenStack compute driver methods ex_create_snapshot and ex_delete_snapshot are now deprecated by the standard methods create_volume_snapshot and destroy_volume_snapshot. You should update your code.
  • The compute base driver now considers the name argument to create_volume_snapshot to be optional. All official implementations of this methods already considered it optional. You should update any custom drivers if they rely on the name being mandatory.
Actions #3

Updated by Peter Amstutz almost 8 years ago

  • Subject changed from [node manager] Test & upgrade to latest libcloud when libcloud merges azure_arm support to [node manager] Upgrade to libcloud 1.4+
Actions #4

Updated by Tom Morris almost 8 years ago

  • Target version changed from Arvados Future Sprints to 2017-02-01 sprint
Actions #5

Updated by Tom Morris almost 8 years ago

  • Target version changed from 2017-02-01 sprint to Arvados Future Sprints
Actions #6

Updated by Peter Amstutz over 7 years ago

  • Subject changed from [node manager] Upgrade to libcloud 1.4+ to [node manager] Upgrade to libcloud 2.0
  • Description updated (diff)
Actions #7

Updated by Tom Morris over 7 years ago

  • Story points set to 2.0
Actions #8

Updated by Tom Morris over 7 years ago

  • Target version changed from Arvados Future Sprints to 2017-08-16 sprint
Actions #9

Updated by Tom Morris over 7 years ago

  • Target version changed from 2017-08-16 sprint to 2017-08-30 Sprint
Actions #10

Updated by Tom Morris over 7 years ago

  • Target version changed from 2017-08-30 Sprint to 2017-09-13 Sprint
Actions #11

Updated by Tom Morris about 7 years ago

  • Assigned To set to Lucas Di Pentima
Actions #12

Updated by Lucas Di Pentima about 7 years ago

  • Status changed from New to In Progress
Actions #13

Updated by Lucas Di Pentima about 7 years ago

This is resolved by our updates on #12055. Right now we're using our 2.2.1.dev1 fork while waiting for libcloud to release a version that includes our fixes regarding tag-on-create feature on AWS compute nodes (https://github.com/apache/libcloud/pull/1100)

Actions #14

Updated by Lucas Di Pentima about 7 years ago

  • Target version changed from 2017-09-13 Sprint to 2017-09-27 Sprint
Actions #15

Updated by Lucas Di Pentima about 7 years ago

  • Status changed from In Progress to New
Actions #16

Updated by Tom Morris about 7 years ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF