Project

General

Profile

Actions

Feature #19961

open

Detect and log spot instance interruption notices

Added by Tom Clegg 9 days ago. Updated 3 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Crunch
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
1.0

Description

When running on a spot instance, crunch-run should check for spot instance interruption notices (AWS recommends checking every 5s) and, when an interruption notice is detected,
  • mention it in the crunch-run.txt log file
  • mention it in the runtime_status container field
  • save the log collection
  • update the container record with the new log pdh
  • continue running (perhaps the container will finish before the instance gets shut down)
  • should have an easy, documented way determining programmatically that this happened

Related issues

Related to Arvados Epics - Story #18179: Better spot instance supportIn Progress03/01/202204/30/2023

Actions
Has duplicate Arvados - Feature #19964: Check for spot instance interruption noticesDuplicate

Actions
Blocks Arvados - Feature #19982: Ability to know when a container died because of spot instance reclamation and option to resubmitNew

Actions
Actions #1

Updated by Tom Clegg 9 days ago

  • Related to Story #18179: Better spot instance support added
Actions #2

Updated by Tom Clegg 8 days ago

  • Has duplicate Feature #19964: Check for spot instance interruption notices added
Actions #3

Updated by Peter Amstutz 3 days ago

  • Blocks Feature #19982: Ability to know when a container died because of spot instance reclamation and option to resubmit added
Actions #4

Updated by Peter Amstutz 3 days ago

  • Description updated (diff)
Actions #5

Updated by Peter Amstutz 3 days ago

  • Target version changed from To be scheduled to 2023-02-15 sprint
Actions

Also available in: Atom PDF