Feature #20983
Updated by Peter Amstutz 7 months ago
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstanceTypeOfferings.html
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/using-ppslong.html
Write a tool that gets all the instance type info from AWS (RAM/CPU spec, included disk, reserved price), filters the list with the options listed below, and print out (or update) the InstanceTypes section of the Arvados config file.
It looks like the price list has the relevant info, e.g.
In the "products" section:
"sku" : "QUMEF4UK3NPT4MN3"
"productFamily" : "Compute Instance",
"vcpu" : "4"
"memory" : "7.5 GiB"
"storage" : "2 x 40 SSD" (I don't know how multiple instance disks are presented to the operating system or if we support this in our current compute image)
"marketoption" : "OnDemand",
In the "terms" "OnDemand" section:
"sku" : "QUMEF4UK3NPT4MN3"
"unit" : "Hrs",
"pricePerUnit" : {
"USD" : "0.3760000000"
},
Include a flag to indicate if ebs autoscale is in use. If EBS autoscale is enabled the disk section should specify 5 TiB of included storage.
If not using EBS autoscale, flag to set default # of GiB of attached storage for "EBS Only" volumes.
Include flag to specify one or more availability zones and get the instance types for those availability zones.
Include flags to specify minimum cores, minimum RAM, and maximum price.
This tool should preferably be written in Go and added as a sub-command of arvados-server.
Example AWS command line for getting instance types in an availability zone (apparently there's other ways of querying for this but they are incredibly slow)
aws ec2 describe-instance-type-offerings --filters "Name=location,Values=us-east-1a" --location-type availability-zone --region us-east-1
If we do end up using
Using DescribeInstanceTypes, here's the relevant fields: use these fields
instanceStorageInfo, memoryInfo, vCpuInfo
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstanceTypes.html