Project

General

Profile

Actions

Feature #19263

open

Support preemptible containers on LSF

Added by Peter Amstutz 3 months ago. Updated 5 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Crunch
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

HPC systems can support low priority / preemptible jobs similar to cloud. One way is by having a separate "preemptible" queue.

Propose that, similar to GPU support, if a job is marked "preemptible", the Arvados configuration can supply a command line fragment that submits the job to a site-specific "preemptible" queue.

The dispatcher must also recognize when a job has been preempted and behave appropriately (e.g. not try to re-submit the job). I.e. when LSF preempts a job, it can be configured to suspend it with SIGSTOP and keep it in in memory to resume later. We don't want Arvados to think a suspended job has failed and try to cancel it.

Actions #1

Updated by Peter Amstutz 3 months ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz 3 months ago

  • Subject changed from Support preemptible on HPC to Support preemptible containers on HPC
Actions #4

Updated by Peter Amstutz 13 days ago

  • Subject changed from Support preemptible containers on HPC to Support preemptible containers on LSF
Actions #5

Updated by Peter Amstutz 13 days ago

  • Target version set to 2022-10-12 sprint
Actions #6

Updated by Peter Amstutz 13 days ago

  • Description updated (diff)
Actions #7

Updated by Peter Amstutz 5 days ago

  • Target version changed from 2022-10-12 sprint to 2022-10-26 sprint
Actions

Also available in: Atom PDF