Project

General

Profile

Idea #8561

Updated by Brett Smith about 8 years ago

Our ping script in cloud nodes ensure that compute nodes set their "ec2_instance_id" when they ping Arvados.    This is a unique identifier for the node in the cloud, although exactly where it comes from in the API varies across this cloud.    It is stored in the info field of the Arvados node record. 

 Node Manager currently compares the IP address of the cloud node record and the Arvados node record to pair nodes.    Determining a node's IP address requires an extra API call in Azure, and is relatively expensive.    Instead, Node Manager should compare the cloud node's unique identifier with the Arvados node's ec2_instance_id.    This may require a new method on cloud node drivers to account for differences across clouds (e.g., on one cloud the identifier is "id" and on another it's "name"). 

 * Add a new class method @node_id(cls, node)@ to the cloud node drivers in @arvnodeman/computenode/driver@. 
 ** In the base driver in @__init__.py@, this should raise NotImplementedError. 
 ** In the EC2 and GCE drivers, this returns @node.id@. 
 ** In the Azure driver, this returns @node.name@. 
 * In @arvnodeman/computenode/dispatch/__init__.py@, in the @offer_arvados_pair@ method, replace the condition @(arvados_node['ip_address'] in self.cloud_node.private_ips)@ with @(arvados_node['info'].get('ec2_instance_id') == self._cloud.node_id(self.cloud_node))@. 

 Set the flag for the Azure libcloud driver to tell it that it no longer needs to query node IP addresses, to improve the performance of Node Manager on Azure. 

 * In Node Manager's Azure compute node driver's list_nodes method, when we call @super(ComputeNodeDriver, self).list_nodes()@, add the argument @ex_fetch_nic=False@ to @list_nodes()@. 

Back