Feature #8019
Updated by Peter Amstutz about 7 years ago
Implement log throttling in crunch-run. Behavior should be equivalent to services/api/lib/crunch_dispatch.rb#rate_limit Use same configuration parameters as currently in API server config: <pre> # These two settings control how frequently log events are flushed to crunchrun support for the following parameters: - limit on # database. Log of lines are buffered until either crunch_log_bytes_per_event # has been reached or crunch_log_seconds_between_events has elapsed since # the last flush. crunch_log_bytes_per_event: 4096 crunch_log_seconds_between_events: 1 # The sample per period for throttling logs, in seconds. crunch_log_throttle_period: 60 # Maximum number - length of bytes that job can log over crunch_log_throttle_period period - total # before being silenced until the end of the period. crunch_log_throttle_bytes: 65536 # Maximum number of lines that per job can log over crunch_log_throttle_period # before being silenced until the end Use same of parameters as used by the period. crunch_log_throttle_lines: 1024 # Maximum bytes that may be logged by a single job. Log bytes that are # silenced by throttling are not counted against this total. crunch_limit_log_bytes_per_job: 67108864 </pre> Above parameters should be API server today, but published in the discovery discover document for use by crunch-run. so that crunchrun can query can