Project

General

Profile

Actions

Bug #21611

open

preemption notices do not appear in crunch-run.txt

Added by Peter Amstutz about 1 month ago. Updated 4 days ago.

Status:
In Progress
Priority:
Normal
Assigned To:
Category:
Crunch
Story points:
-

Description

I've looked at a number of containers now that have been preempted and none of them have crunch-run.txt updated to say that it received a preemption notice even though it is supposed to.


Subtasks 1 (1 open0 closed)

Task #21734: ReviewNewPeter AmstutzActions
Actions #1

Updated by Peter Amstutz 25 days ago

  • Target version changed from Future to Development 2024-04-24 sprint
Actions #2

Updated by Peter Amstutz 25 days ago

  • Description updated (diff)
Actions #3

Updated by Peter Amstutz 19 days ago

  • Target version changed from Development 2024-04-24 sprint to Development 2024-05-08 sprint
Actions #4

Updated by Peter Amstutz 19 days ago

  • Target version changed from Development 2024-05-08 sprint to Development 2024-04-24 sprint
Actions #5

Updated by Peter Amstutz 19 days ago

  • Target version changed from Development 2024-04-24 sprint to Development 2024-05-08 sprint
Actions #6

Updated by Peter Amstutz 4 days ago

  • Description updated (diff)
  • Subject changed from crunch-run updates copy of container.json in log collection when a container ends and/or runtime_status is updated to preemption notices do not appear in crunch-run.txt
  • Tracker changed from Feature to Bug
Actions #7

Updated by Peter Amstutz 4 days ago

  • Assigned To set to Tom Clegg
Actions #8

Updated by Tom Clegg 4 days ago

  • Status changed from New to In Progress
I suspect the "write log, then save log collection" is doing the opposite of what we want, because
  • "write log entry" just writes the message to the throttled-logging buffer, not yet to the log collection
  • "save log collection" saves the log collection and resets the auto-flush timer, minimizing the chance auto-flush will happen before the preempted instance shuts down

It's possible something else is going on too, but either way, we should rearrange the logging pipeline so the log collection gets updated immediately instead of after the "group logs into chunks" step. If nothing else, that will reduce latency for showing logs in workbench.

Actions

Also available in: Atom PDF