Story #14931
closed
[arvados-dispatch-cloud] Configurable instance tags
100%
Description
- Admin can supply some tags that get set on all instances created by arvados-dispatch-cloud (e.g., {"myOrg-instance-class": "arvados", "admin-contact": "bob@example.com"})
- Admin can control the tag prefix (currently a prefix is hard-coded by each driver, but this should be moved out to the dispatcher, and the driver should just do exactly what it's told)
Instance-specific resources (like NICs in Azure) should also be tagged, using the same tags as the instance.
Shared resources (like key pairs in EC2) should be tagged with the preconfigured tags, but not the per-instance tags. (Implementation: the driver doesn't know which tags are instance-specific, so the set of tags to use for shared resources should be supplied explicitly during driver initialization.)
Tag keys and values are case sensitive.
Related issues
Updated by Tom Clegg over 3 years ago
- Related to Feature #14291: [crunch-dispatch-cloud] AWS driver added
Updated by Tom Clegg over 3 years ago
- Related to Story #13908: [Epic] Replace SLURM for cloud job scheduling/dispatching added
Updated by Tom Morris about 3 years ago
- Target version changed from To Be Groomed to Arvados Future Sprints
- Story points set to 1.0
Updated by Tom Morris about 3 years ago
- Related to Story #15075: [a-d-c] Support for extra tags on resources created by arvados added
Updated by Tom Clegg about 3 years ago
- Has duplicate Story #15075: [a-d-c] Support for extra tags on resources created by arvados added
Updated by Tom Clegg about 3 years ago
- Related to deleted (Story #15075: [a-d-c] Support for extra tags on resources created by arvados )
Updated by Tom Clegg about 3 years ago
- Status changed from New to In Progress
- Assigned To set to Tom Clegg
- Target version changed from Arvados Future Sprints to 2019-06-05 Sprint
Updated by Tom Clegg about 3 years ago
- Related to Feature #15063: [a-d-c] Assign names to EC2 instances added
Updated by Tom Clegg about 3 years ago
Updated by Peter Amstutz about 3 years ago
I think the default value of TagKeyPrefix in config.yml.default should be "arvados-" instead of blank.
It looks like you can set a static value for a tag like "name" (so you can have a bunch of instances called "arvados compute" which is better than being blank) but do we want a way to set the name based on the Arvados instance id? I'm thinking of the case where the admin needs to correlate an instance on the native cloud dashboard with cloud-dispatch logs. Perhaps the tag value could be interpreted as a format string?
Updated by Tom Clegg about 3 years ago
Peter Amstutz wrote:
I think the default value of TagKeyPrefix in config.yml.default should be "arvados-" instead of blank.
How about Arvados, so we get ArvadosInstanceType etc.?
It looks like you can set a static value for a tag like "name" (so you can have a bunch of instances called "arvados compute" which is better than being blank) but do we want a way to set the name based on the Arvados instance id? I'm thinking of the case where the admin needs to correlate an instance on the native cloud dashboard with cloud-dispatch logs. Perhaps the tag value could be interpreted as a format string?
Dispatch-cloud uses the provider-assigned instance ID instead of inventing its own for logs, status, etc., so I don't think an additional cross-reference field is needed.
Also, that provider-assigned instance ID isn't known yet when we supply the initial tags for a new instance, so we'd have to start with a fake value and then change it on next sync, which would create edge cases for anyone trying to use it.
Updated by Peter Amstutz about 3 years ago
Tom Clegg wrote:
Peter Amstutz wrote:
I think the default value of TagKeyPrefix in config.yml.default should be "arvados-" instead of blank.
How about Arvados, so we get ArvadosInstanceType etc.?
Sounds good to me.
It looks like you can set a static value for a tag like "name" (so you can have a bunch of instances called "arvados compute" which is better than being blank) but do we want a way to set the name based on the Arvados instance id? I'm thinking of the case where the admin needs to correlate an instance on the native cloud dashboard with cloud-dispatch logs. Perhaps the tag value could be interpreted as a format string?
Dispatch-cloud uses the provider-assigned instance ID instead of inventing its own for logs, status, etc., so I don't think an additional cross-reference field is needed.
Also, that provider-assigned instance ID isn't known yet when we supply the initial tags for a new instance, so we'd have to start with a fake value and then change it on next sync, which would create edge cases for anyone trying to use it.
Sorry about that, I was thinking of Azure where the instance ids are made up by the driver. You're right, so long as the instance ids in the logs match up with the instance ids on the cloud's dashboard, there is nothing to do here.
Please merge after updating the TagKeyPrefix default.
Updated by Tom Clegg about 3 years ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|c792e4991e1d77620d61efaa2600a93d75227f06.