Feature #18596
Config option to enable preemptible variants of all instance types
100%
Description
Currently, in order to use preemptible instances, the operator needs to add InstanceTypes entries for all instance types they hope to use, with the Preemptible flag set, and possibly a lower price. This is tedious.
In most cases it would be much more convenient to have a config option (PreemptiblePriceFactor?) that automatically duplicates every non-preemptible InstanceTypes entry that does not already have a preemptible counterpart, with a max/bid price of PreemptiblePriceFactor × Price of the non-preemptible variant.
It will be the job of the go config loader to add the additional preemptible variants to instance types.
Subtasks
Related issues
History
#1
Updated by Tom Clegg 5 months ago
Updated by - Related to Bug #18562: [api] should not change the preemptible flag across the board added
#3
Updated by Ward Vandewege 4 months ago
Updated by - Related to Story #18179: Better spot instance support added
#4
Updated by Peter Amstutz 3 months ago
Updated by - Target version set to 2022-03-16 sprint
- Description updated (diff)
#6
Updated by Peter Amstutz 2 months ago
- Target version changed from 2022-03-16 sprint to 2022-03-30 Sprint
#7
Updated by Tom Clegg 2 months ago
18596-preemptible-price-factor @ 7b2c2c415aa67936389998a5c93d52891e5644f4 -- developer-run-tests: #2972
#8
Updated by Tom Clegg 2 months ago
What should we do if the config has a non-zero PreemptiblePriceFactor and instance type "x" and instance type "x.preemptible" that differs from the one we'd add automatically? Warning and use the explicitly configured one? (Current branch will overwrite with the auto-generated one.)
#9
Updated by Ward Vandewege 2 months ago
- Status changed from New to In Progress
#10
Updated by Ward Vandewege 2 months ago
Tom Clegg wrote:
18596-preemptible-price-factor @ 7b2c2c415aa67936389998a5c93d52891e5644f4 -- developer-run-tests: #2972
A few things:
- https://doc.arvados.org/admin/spot-instances.html needs to be updated to reflect this new option. I see that that page also incorrectly states that
AlwaysUsePreemptibleInstancesdefaults tofalse.
- in autofillPreemptible, any existing preemptible node types that match our naming pattern will be silently overwritten. Should this be a
arvados-server config-checkwarning?
- in
lib/config/config.default.yml, is it worth pointing out that settingPreemptiblePriceFactorto 1 could be a good choice?
Thanks for that AnonymousToken bugfix! LGTM otherwise.
#11
Updated by Tom Clegg 2 months ago
wb1 retry developer-run-tests-apps-workbench-integration: #3175
- update admin doc page
- change AlwaysUse default to false to match docs (looks like that was intended but missed in #18180)
- if auto preemptible instance would overwrite explicitly configured "{name}.preemptible", warn instead of overwriting it
- mention in the config template that factor 1.0 is a reasonable choice
#12
Updated by Ward Vandewege 2 months ago
Tom Clegg wrote:
18596-preemptible-price-factor @ 2b0dc5ddea4e88a4fcf699b63b720841f77b76c5 -- developer-run-tests: #2976
wb1 retry developer-run-tests-apps-workbench-integration: #3175
- update admin doc page
- change AlwaysUse default to false to match docs (looks like that was intended but missed in #18180)
- if auto preemptible instance would overwrite explicitly configured "{name}.preemptible", warn instead of overwriting it
- mention in the config template that factor 1.0 is a reasonable choice
Thank you! This LGTM.
#14
Updated by Peter Amstutz 2 months ago
- Release set to 46
