Story #14931
[arvados-dispatch-cloud] Configurable instance tags
100%
Description
- Admin can supply some tags that get set on all instances created by arvados-dispatch-cloud (e.g., {"myOrg-instance-class": "arvados", "admin-contact": "bob@example.com"})
- Admin can control the tag prefix (currently a prefix is hard-coded by each driver, but this should be moved out to the dispatcher, and the driver should just do exactly what it's told)
Instance-specific resources (like NICs in Azure) should also be tagged, using the same tags as the instance.
Shared resources (like key pairs in EC2) should be tagged with the preconfigured tags, but not the per-instance tags. (Implementation: the driver doesn't know which tags are instance-specific, so the set of tags to use for shared resources should be supplied explicitly during driver initialization.)
Tag keys and values are case sensitive.
Subtasks
Related issues
Associated revisions
History
#1
Updated by Tom Clegg almost 2 years ago
- Related to Feature #14291: [crunch-dispatch-cloud] AWS driver added
#2
Updated by Tom Clegg almost 2 years ago
- Related to Story #13908: [Epic] Replace SLURM for cloud job scheduling/dispatching added
#3
Updated by Tom Morris almost 2 years ago
- Tracker changed from Bug to Story
#4
Updated by Tom Morris almost 2 years ago
- Target version changed from To Be Groomed to Arvados Future Sprints
- Story points set to 1.0
#5
Updated by Tom Morris almost 2 years ago
- Related to Story #15075: [a-d-c] Support for extra tags on resources created by arvados added
#6
Updated by Tom Clegg almost 2 years ago
- Description updated (diff)
#7
Updated by Tom Clegg almost 2 years ago
- Description updated (diff)
#8
Updated by Tom Clegg almost 2 years ago
- Has duplicate Story #15075: [a-d-c] Support for extra tags on resources created by arvados added
#9
Updated by Tom Clegg almost 2 years ago
- Related to deleted (Story #15075: [a-d-c] Support for extra tags on resources created by arvados )
#10
Updated by Tom Clegg over 1 year ago
- Status changed from New to In Progress
- Assigned To set to Tom Clegg
- Target version changed from Arvados Future Sprints to 2019-06-05 Sprint
#11
Updated by Tom Clegg over 1 year ago
- Related to Feature #15063: [a-d-c] Assign names to EC2 instances added
#12
Updated by Tom Clegg over 1 year ago
#13
Updated by Peter Amstutz over 1 year ago
I think the default value of TagKeyPrefix in config.yml.default should be "arvados-" instead of blank.
It looks like you can set a static value for a tag like "name" (so you can have a bunch of instances called "arvados compute" which is better than being blank) but do we want a way to set the name based on the Arvados instance id? I'm thinking of the case where the admin needs to correlate an instance on the native cloud dashboard with cloud-dispatch logs. Perhaps the tag value could be interpreted as a format string?
#14
Updated by Tom Clegg over 1 year ago
Peter Amstutz wrote:
I think the default value of TagKeyPrefix in config.yml.default should be "arvados-" instead of blank.
How about Arvados, so we get ArvadosInstanceType etc.?
It looks like you can set a static value for a tag like "name" (so you can have a bunch of instances called "arvados compute" which is better than being blank) but do we want a way to set the name based on the Arvados instance id? I'm thinking of the case where the admin needs to correlate an instance on the native cloud dashboard with cloud-dispatch logs. Perhaps the tag value could be interpreted as a format string?
Dispatch-cloud uses the provider-assigned instance ID instead of inventing its own for logs, status, etc., so I don't think an additional cross-reference field is needed.
Also, that provider-assigned instance ID isn't known yet when we supply the initial tags for a new instance, so we'd have to start with a fake value and then change it on next sync, which would create edge cases for anyone trying to use it.
#15
Updated by Peter Amstutz over 1 year ago
Tom Clegg wrote:
Peter Amstutz wrote:
I think the default value of TagKeyPrefix in config.yml.default should be "arvados-" instead of blank.
How about Arvados, so we get ArvadosInstanceType etc.?
Sounds good to me.
It looks like you can set a static value for a tag like "name" (so you can have a bunch of instances called "arvados compute" which is better than being blank) but do we want a way to set the name based on the Arvados instance id? I'm thinking of the case where the admin needs to correlate an instance on the native cloud dashboard with cloud-dispatch logs. Perhaps the tag value could be interpreted as a format string?
Dispatch-cloud uses the provider-assigned instance ID instead of inventing its own for logs, status, etc., so I don't think an additional cross-reference field is needed.
Also, that provider-assigned instance ID isn't known yet when we supply the initial tags for a new instance, so we'd have to start with a fake value and then change it on next sync, which would create edge cases for anyone trying to use it.
Sorry about that, I was thinking of Azure where the instance ids are made up by the driver. You're right, so long as the instance ids in the logs match up with the instance ids on the cloud's dashboard, there is nothing to do here.
Please merge after updating the TagKeyPrefix default.
#16
Updated by Tom Clegg over 1 year ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|c792e4991e1d77620d61efaa2600a93d75227f06.
#17
Updated by Peter Amstutz about 1 year ago
- Release set to 22
Merge branch '14931-custom-tags'
closes #14931
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>