Project

General

Profile

Actions

Feature #4935

closed

[Crunch] Choose worker nodes where the desired docker image is already available.

Added by Bryan Cosca over 9 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
-
Category:
Crunch
Target version:
-
Story points:
-

Description

The scenario where user A is running jobs and debugging a pipeline to work on arvados. User A ran his job and it ran on compute 1. Then he realized something failed, and five minutes later he fixes the problem and commits the change. User A then runs his job again and it runs again on compute1 and fails again and fixes the problem. Then, he runs his job again and realizes it doesn't fail right away. The compute node has changed from under him and he is now using compute2. He now has to wait ~7 minutes for his docker image to load into the node or he has to wait ~10 minutes for the node to spin up and then load his image. His ~1 minute turn around time for development has just become ~10 minutes, leaving the user to simply wait.

If its possible, I would like to develop on one node and not have these nodes pulled from under me if I'm not using them for ~5-10 minutes.

Actions #1

Updated by Tom Clegg over 9 years ago

  • Subject changed from Dedicated Nodes to users to [Crunch] Choose worker nodes where the desired docker image is already available.
Actions #2

Updated by Brett Smith about 9 years ago

  • Category set to Crunch
Actions #3

Updated by Bryan Cosca about 9 years ago

A really good example imo is this instance: qr1hi-d1hrv-sbjgb8za1dpp5kc. The first two jobs ran fine on compute16 and then for some reason, the third job chose compute43 which had #5006 problem.

Actions #4

Updated by Tom Clegg about 9 years ago

  • Target version set to Arvados Future Sprints
Actions #5

Updated by Ward Vandewege almost 3 years ago

  • Target version deleted (Arvados Future Sprints)
Actions #6

Updated by Peter Amstutz about 2 years ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF