Project

General

Profile

Actions

Task #4580

closed

Feature #4138: [Node Manager] Support Google Cloud Platform in node manager

Review 4138-node-manager-google-cloud

Added by Tim Pierce over 9 years ago. Updated about 9 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
Target version:

Description

Ready to review at ab8acfe

Some implementation notes:

  • This code has not been tested against a live GCE account. We can test when we get GCE credentials that allow us to spin nodes up and down.
  • GCE does not permit filtering instances on tags the way AWS does, so filtering has not been implemented. Clients will need to list all instances and do their own filtering.
  • GCE does implement a 'tag' concept, but it is a flat keyword list rather than a key/value structure. When a node is synced, the Arvados hostname is updated on the node by setting a tag 'hostname-arvados.fqdn.name'.
    • The underlying GCE library supports a setMetadata call that should allow us to use metadata to track this in a cleaner way, but the libcloud API doesn't support this call yet.
  • GCE node access is controlled with "service accounts" rather than security groups. These can be defined in gce.example.cfg by setting the service_account setting to a JSON string with the desired service account list.

For reviewing: the GCE drivers and tests were developed by ripping out everything in the EC2 drivers that doesn't look like GCE. Accordingly, I strongly recommend:

diff services/nodemanager/arvnodeman/computenode/driver/ec2.py services/nodemanager/arvnodeman/computenode/driver/gce.py
diff services/nodemanager/tests/test_computenode_driver_ec2.py services/nodemanager/tests/test_computenode_driver_gce.py
diff services/nodemanager/doc/ec2.example.cfg services/nodemanager/doc/gce.example.cfg
Actions #1

Updated by Brett Smith over 9 years ago

  • Status changed from New to In Progress
  • Assigned To set to Brett Smith
Actions #2

Updated by Brett Smith over 9 years ago

  • Assigned To changed from Brett Smith to Tim Pierce
Actions #3

Updated by Tim Pierce about 9 years ago

  • Assigned To changed from Tim Pierce to Ward Vandewege
Actions #4

Updated by Tim Pierce about 9 years ago

  • Assigned To changed from Ward Vandewege to Brett Smith
Actions #5

Updated by Brett Smith about 9 years ago

  • Status changed from In Progress to Closed
  • Remaining (hours) set to 0.0
Actions

Also available in: Atom PDF