Story #8999

[node manager] Upgrade to libcloud 2.0

Added by Peter Amstutz over 1 year ago. Updated 10 days ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:Lucas Di Pentima% Done:

0%

Category:-
Target version:2017-09-27 Sprint
Story points2.0
Velocity based estimate-

Description

The pending libcloud 2.0.0 release has all our work relating to Azure ARM (including cloud environments) merged. We should test to see if node manager works with libcloud 2.0.0 so we can depend the upstream packaged version instead of our own fork.


Related issues

Related to Arvados - Bug #12055: [node manager] ec2 set tags on create Resolved 08/16/2017
Blocked by Arvados - Story #11545: Create a CWL stress test for node manager Resolved 04/25/2017

History

#1 Updated by Brett Smith over 1 year ago

  • Target version set to Arvados Future Sprints

#2 Updated by Tom Morris 10 months ago

Azure ARM support was introduced in libcloud 1.4 which was released at the end of November 2016. We should probably let it settle for a bit to make sure there are no issues with the release before contemplating updating.

We are currently using a 0.18 derivative from summer 2015.

There is a small set of upgrade notes available at http://libcloud.readthedocs.io/en/latest/upgrade_notes.html

Libcloud 1.0.0

  • Per-region provider constants and related driver classes which have been deprecated in Libcloud 0.14.0 have now been fully removed.

    Those provider drivers have moved to the single provider constant + region constructor argument in Libcloud 0.14.0.

Libcloud 0.20.0

  • New optional ttl argument has been added to libcloud.dns.base.Record class constructor before the existing extra argument.

    If you have previously manually instantiated this class and didn’t use keyword arguments, you need to update your code to correctly pass arguments to the constructor (you are encouraged to use keyword arguments to avoid such issues in the future).

  • All NodeState, StorageVolumeState, VolumeSnapshotState and Provider attributes are now strings instead of integers.

    If you are using the tostring and fromstring methods of NodeState, you are fine. If you are using NodeState.RUNNING and the like, you are also fine.

    However, if you have previously depended on these being integers, you need to update your code to depend on strings. You should consider starting using the tostring and fromstring methods as the output of these functions will not change in future versions, while the implementation might.

Libcloud 0.19.0

  • The base signature of NodeDriver.create_volume has changed. The snapshot argument is now expected to be a VolumeSnapshot instead of a string. The older signature was never correct for built-in drivers, but custom drivers may break. (GCE accepted strings, names or None and still does. Other drivers did not implement creating volumes from snapshots at all until now.)
  • VolumeSnapshots now have a created attribute that is a datetime field showing the creation datetime of the snapshot. The field in VolumeSnapshot.extra containing the original string is maintained, so this is a backwards-compatible change.
  • The OpenStack compute driver methods ex_create_snapshot and ex_delete_snapshot are now deprecated by the standard methods create_volume_snapshot and destroy_volume_snapshot. You should update your code.
  • The compute base driver now considers the name argument to create_volume_snapshot to be optional. All official implementations of this methods already considered it optional. You should update any custom drivers if they rely on the name being mandatory.

#3 Updated by Peter Amstutz 9 months ago

  • Subject changed from [node manager] Test & upgrade to latest libcloud when libcloud merges azure_arm support to [node manager] Upgrade to libcloud 1.4+

#4 Updated by Tom Morris 8 months ago

  • Target version changed from Arvados Future Sprints to 2017-02-01 sprint

#5 Updated by Tom Morris 8 months ago

  • Target version changed from 2017-02-01 sprint to Arvados Future Sprints

#6 Updated by Peter Amstutz 5 months ago

  • Subject changed from [node manager] Upgrade to libcloud 1.4+ to [node manager] Upgrade to libcloud 2.0
  • Description updated (diff)

#7 Updated by Tom Morris 5 months ago

  • Story points set to 2.0

#8 Updated by Tom Morris about 1 month ago

  • Target version changed from Arvados Future Sprints to 2017-08-16 sprint

#9 Updated by Tom Morris about 1 month ago

  • Target version changed from 2017-08-16 sprint to 2017-08-30 Sprint

#10 Updated by Tom Morris about 1 month ago

  • Target version changed from 2017-08-30 Sprint to 2017-09-13 Sprint

#11 Updated by Tom Morris 25 days ago

  • Assignee set to Lucas Di Pentima

#12 Updated by Lucas Di Pentima 20 days ago

  • Status changed from New to In Progress

#13 Updated by Lucas Di Pentima 20 days ago

This is resolved by our updates on #12055. Right now we're using our 2.2.1.dev1 fork while waiting for libcloud to release a version that includes our fixes regarding tag-on-create feature on AWS compute nodes (https://github.com/apache/libcloud/pull/1100)

#14 Updated by Lucas Di Pentima 10 days ago

  • Target version changed from 2017-09-13 Sprint to 2017-09-27 Sprint

#15 Updated by Lucas Di Pentima 10 days ago

  • Status changed from In Progress to New

#16 Updated by Tom Morris 10 days ago

  • Status changed from New to Closed

Also available in: Atom PDF