Story #7980

[Node Manager] Broken nodes always get shut down, exempt from all node-counting considerations

Added by Brett Smith almost 5 years ago. Updated 21 days ago.

Status:
Closed
Priority:
Normal
Assigned To:
-
Category:
Node Manager
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

Right now we're in a situation where we want the workaround from #7286 to kick in, but the daemon is declining to shut the nodes down because there are jobs in the queue waiting to be run. Shutdowns caused by node malfunction—basically, any case in ComputeNodeMonitorActor.shutdown_eligible except for self.in_state('idle')—should be respected unconditionally.

This will probably involve having consider_shutdown send a "force" boolean to its subscribers, and updating the daemon to respect that.

History

#1 Updated by Brett Smith almost 5 years ago

  • Description updated (diff)
  • Category set to Node Manager

#2 Updated by Brett Smith almost 5 years ago

  • Target version set to Arvados Future Sprints

#3 Updated by Ward Vandewege 21 days ago

  • Status changed from New to Closed

Also available in: Atom PDF