Idea #14931
closed
[arvados-dispatch-cloud] Configurable instance tags
Added by Tom Clegg about 5 years ago.
Updated about 4 years ago.
Release relationship:
Auto
Description
Two related features would help make arvados-dispatch-cloud play nicer with other ways instance tags are used in the same cloud account:
- Admin can supply some tags that get set on all instances created by arvados-dispatch-cloud (e.g., {"myOrg-instance-class": "arvados", "admin-contact": "bob@example.com"})
- Admin can control the tag prefix (currently a prefix is hard-coded by each driver, but this should be moved out to the dispatcher, and the driver should just do exactly what it's told)
Instance-specific resources (like NICs in Azure) should also be tagged, using the same tags as the instance.
Shared resources (like key pairs in EC2) should be tagged with the preconfigured tags, but not the per-instance tags. (Implementation: the driver doesn't know which tags are instance-specific, so the set of tags to use for shared resources should be supplied explicitly during driver initialization.)
Tag keys and values are case sensitive.
- Related to Idea #13908: [Epic] Replace SLURM for cloud job scheduling/dispatching added
- Tracker changed from Bug to Idea
- Target version changed from To Be Groomed to Arvados Future Sprints
- Story points set to 1.0
- Related to Idea #15075: [a-d-c] Support for extra tags on resources created by arvados added
- Description updated (diff)
- Description updated (diff)
- Has duplicate Idea #15075: [a-d-c] Support for extra tags on resources created by arvados added
- Related to deleted (Idea #15075: [a-d-c] Support for extra tags on resources created by arvados )
- Status changed from New to In Progress
- Assigned To set to Tom Clegg
- Target version changed from Arvados Future Sprints to 2019-06-05 Sprint
- Related to Feature #15063: [a-d-c] Assign names to EC2 instances added
I think the default value of TagKeyPrefix in config.yml.default should be "arvados-" instead of blank.
It looks like you can set a static value for a tag like "name" (so you can have a bunch of instances called "arvados compute" which is better than being blank) but do we want a way to set the name based on the Arvados instance id? I'm thinking of the case where the admin needs to correlate an instance on the native cloud dashboard with cloud-dispatch logs. Perhaps the tag value could be interpreted as a format string?
Peter Amstutz wrote:
I think the default value of TagKeyPrefix in config.yml.default should be "arvados-" instead of blank.
How about Arvados, so we get ArvadosInstanceType etc.?
It looks like you can set a static value for a tag like "name" (so you can have a bunch of instances called "arvados compute" which is better than being blank) but do we want a way to set the name based on the Arvados instance id? I'm thinking of the case where the admin needs to correlate an instance on the native cloud dashboard with cloud-dispatch logs. Perhaps the tag value could be interpreted as a format string?
Dispatch-cloud uses the provider-assigned instance ID instead of inventing its own for logs, status, etc., so I don't think an additional cross-reference field is needed.
Also, that provider-assigned instance ID isn't known yet when we supply the initial tags for a new instance, so we'd have to start with a fake value and then change it on next sync, which would create edge cases for anyone trying to use it.
Tom Clegg wrote:
Peter Amstutz wrote:
I think the default value of TagKeyPrefix in config.yml.default should be "arvados-" instead of blank.
How about Arvados, so we get ArvadosInstanceType etc.?
Sounds good to me.
It looks like you can set a static value for a tag like "name" (so you can have a bunch of instances called "arvados compute" which is better than being blank) but do we want a way to set the name based on the Arvados instance id? I'm thinking of the case where the admin needs to correlate an instance on the native cloud dashboard with cloud-dispatch logs. Perhaps the tag value could be interpreted as a format string?
Dispatch-cloud uses the provider-assigned instance ID instead of inventing its own for logs, status, etc., so I don't think an additional cross-reference field is needed.
Also, that provider-assigned instance ID isn't known yet when we supply the initial tags for a new instance, so we'd have to start with a fake value and then change it on next sync, which would create edge cases for anyone trying to use it.
Sorry about that, I was thinking of Azure where the instance ids are made up by the driver. You're right, so long as the instance ids in the logs match up with the instance ids on the cloud's dashboard, there is nothing to do here.
Please merge after updating the TagKeyPrefix default.
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Also available in: Atom
PDF
Updated by 
Updated by Tom Morris 
Updated by