Task #4580

Feature #4138: [Node Manager] Support Google Cloud Platform in node manager

Review 4138-node-manager-google-cloud

Added by Tim Pierce over 5 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
Brett Smith
Category:
-
Target version:
Start date:
11/17/2014
Due date:
% Done:

0%

Estimated time:

Description

Ready to review at ab8acfe

Some implementation notes:

  • This code has not been tested against a live GCE account. We can test when we get GCE credentials that allow us to spin nodes up and down.
  • GCE does not permit filtering instances on tags the way AWS does, so filtering has not been implemented. Clients will need to list all instances and do their own filtering.
  • GCE does implement a 'tag' concept, but it is a flat keyword list rather than a key/value structure. When a node is synced, the Arvados hostname is updated on the node by setting a tag 'hostname-arvados.fqdn.name'.
    • The underlying GCE library supports a setMetadata call that should allow us to use metadata to track this in a cleaner way, but the libcloud API doesn't support this call yet.
  • GCE node access is controlled with "service accounts" rather than security groups. These can be defined in gce.example.cfg by setting the service_account setting to a JSON string with the desired service account list.

For reviewing: the GCE drivers and tests were developed by ripping out everything in the EC2 drivers that doesn't look like GCE. Accordingly, I strongly recommend:

diff services/nodemanager/arvnodeman/computenode/driver/ec2.py services/nodemanager/arvnodeman/computenode/driver/gce.py
diff services/nodemanager/tests/test_computenode_driver_ec2.py services/nodemanager/tests/test_computenode_driver_gce.py
diff services/nodemanager/doc/ec2.example.cfg services/nodemanager/doc/gce.example.cfg

History

#1 Updated by Brett Smith over 5 years ago

  • Status changed from New to In Progress
  • Assigned To set to Brett Smith

#2 Updated by Brett Smith over 5 years ago

  • Assigned To changed from Brett Smith to Tim Pierce

#3 Updated by Tim Pierce over 5 years ago

  • Assigned To changed from Tim Pierce to Ward Vandewege

#4 Updated by Tim Pierce over 5 years ago

  • Assigned To changed from Ward Vandewege to Brett Smith

#5 Updated by Brett Smith over 5 years ago

  • Status changed from In Progress to Closed
  • Remaining (hours) set to 0.0

Also available in: Atom PDF