Project

General

Profile

Bug #2934

Updated by Tom Clegg almost 10 years ago

Current code limits each job to one Log entry per second. Additional limits to impose: 
 * 64K Log entries per job 
 * 64Mbytes of log data per job 

 When one of these limits is reached, add one more entry like @"Server configured limit reached (crunch_limit_log_events_per_job: 65536). Subsequent logs truncated"@. 

 Subsequent logs will still get stored in Keep, but won't be available through event bus. 

 These limits (and the current @LOG_BUFFER_SIZE@ constant) should be configurable. Add to @services/api/config/application.default.yml@ in the @common@ section: 

 <pre> 
   crunch_limit_log_events_per_job: 65536 
   crunch_limit_log_event_bytes_per_job: 67108864 
   crunch_limit_log_bytes_per_event: 4096 
 </pre> 

 Then use @Rails.configuration.crunch_limit_log_bytes_per_event@ instead of @LOG_BUFFER_SIZE@, etc. 

 Optional: Add @crunch_limit_seconds_between_log_events@ (this is a minimum, and currently hard coded at 1, but we can probably support 0, 0.5, 5, etc., easily enough). 

Back