Feature #7399
Updated by Brett Smith over 8 years ago
h2. Bug report With the addition of log throttling (Exceeded rate 65536 bytes per 60 seconds (crunch_log_throttle_bytes). Logging will be silenced for the next 16 seconds.) I can lose a lot of useful logs in that time window. My theory is to reduce the logging of crunchstat in order to see more what is going on in my computation without that clutter. If logging crunchstat does not add much to the rate, it might be useful to have a way to increase the crunch_log_throttle_bytes from an SDK or something. Basically, I want a feature to be able to see all the log output without throttling (but I know that was impossible before crunch_log_throttle) h2. Implementation * Log throttling only applies to live logs, not the logs stored in Keep. Improve the message logs when jobs get throttled to help clarify this. * When crunch-dispatch reads \r not followed by \n, report that a progress bar was detected and proactively throttle those logs. Resume normal logging (subject to the normal throttling limits) after \n is read again.