Project

General

Profile

Actions

Feature #18596

closed

Config option to enable preemptible variants of all instance types

Added by Tom Clegg about 1 year ago. Updated 11 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
03/21/2022
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

Currently, in order to use preemptible instances, the operator needs to add InstanceTypes entries for all instance types they hope to use, with the Preemptible flag set, and possibly a lower price. This is tedious.

In most cases it would be much more convenient to have a config option (PreemptiblePriceFactor?) that automatically duplicates every non-preemptible InstanceTypes entry that does not already have a preemptible counterpart, with a max/bid price of PreemptiblePriceFactor × Price of the non-preemptible variant.

It will be the job of the go config loader to add the additional preemptible variants to instance types.


Subtasks 1 (0 open1 closed)

Task #18816: review 18596-preemptible-price-factorResolvedWard Vandewege03/21/2022

Actions

Related issues

Related to Arvados - Bug #18562: [api] should not change the preemptible flag across the boardResolvedTom Clegg12/23/2021

Actions
Related to Arvados Epics - Story #18179: Better spot instance supportIn Progress03/01/202204/30/2023

Actions
Actions #1

Updated by Tom Clegg about 1 year ago

  • Related to Bug #18562: [api] should not change the preemptible flag across the board added
Actions #2

Updated by Tom Clegg about 1 year ago

  • Subject changed from Config option enable preemptible variants of all instance types to Config option to enable preemptible variants of all instance types
Actions #3

Updated by Ward Vandewege about 1 year ago

  • Related to Story #18179: Better spot instance support added
Actions #4

Updated by Peter Amstutz 12 months ago

  • Target version set to 2022-03-16 sprint
  • Description updated (diff)
Actions #5

Updated by Tom Clegg 11 months ago

  • Assigned To set to Tom Clegg
Actions #6

Updated by Peter Amstutz 11 months ago

  • Target version changed from 2022-03-16 sprint to 2022-03-30 Sprint
Actions #7

Updated by Tom Clegg 11 months ago

Actions #8

Updated by Tom Clegg 11 months ago

What should we do if the config has a non-zero PreemptiblePriceFactor and instance type "x" and instance type "x.preemptible" that differs from the one we'd add automatically? Warning and use the explicitly configured one? (Current branch will overwrite with the auto-generated one.)

Actions #9

Updated by Ward Vandewege 11 months ago

  • Status changed from New to In Progress
Actions #10

Updated by Ward Vandewege 11 months ago

Tom Clegg wrote:

18596-preemptible-price-factor @ 7b2c2c415aa67936389998a5c93d52891e5644f4 -- developer-run-tests: #2972

A few things:

  • in autofillPreemptible, any existing preemptible node types that match our naming pattern will be silently overwritten. Should this be a arvados-server config-check warning?
  • in lib/config/config.default.yml, is it worth pointing out that setting PreemptiblePriceFactor to 1 could be a good choice?

Thanks for that AnonymousToken bugfix! LGTM otherwise.

Actions #11

Updated by Tom Clegg 11 months ago

18596-preemptible-price-factor @ 2b0dc5ddea4e88a4fcf699b63b720841f77b76c5 -- developer-run-tests: #2976
wb1 retry developer-run-tests-apps-workbench-integration: #3175
  • update admin doc page
  • change AlwaysUse default to false to match docs (looks like that was intended but missed in #18180)
  • if auto preemptible instance would overwrite explicitly configured "{name}.preemptible", warn instead of overwriting it
  • mention in the config template that factor 1.0 is a reasonable choice
Actions #12

Updated by Ward Vandewege 11 months ago

Tom Clegg wrote:

18596-preemptible-price-factor @ 2b0dc5ddea4e88a4fcf699b63b720841f77b76c5 -- developer-run-tests: #2976
wb1 retry developer-run-tests-apps-workbench-integration: #3175
  • update admin doc page
  • change AlwaysUse default to false to match docs (looks like that was intended but missed in #18180)
  • if auto preemptible instance would overwrite explicitly configured "{name}.preemptible", warn instead of overwriting it
  • mention in the config template that factor 1.0 is a reasonable choice

Thank you! This LGTM.

Actions #13

Updated by Tom Clegg 11 months ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados-private:commit:arvados|926f71e8481d62a20ff66a834d876713df5fd0c7.

Actions #14

Updated by Peter Amstutz 11 months ago

  • Release set to 46
Actions

Also available in: Atom PDF