Project

General

Profile

Bug #20183

Updated by Tom Clegg about 1 year ago

Currently, the UpdatePriority.run_update_thread hook appears to be associated with unusually high memory usage in passenger workers processes. 

 Proposed fix is to handle these asynchronous updates using a rails handler called by a controller thread, similar to trash sweeps. 

 After a container create/update/delete call, controller should 
 * acquire a (new) dblock.DBLocker, @UpdateContainerPriority@, waiting if necessary 
 * check for "priority>0 but should be 0" and "priority=0 but should be >0" using queries like the ones in the current ruby code source:services/api/lib/update_priority.go 
 * if any need fixing, controller should call rails to fix them in batches, using a (new) system-only endpoint @update_priority@ 
 * release @UpdateContainerPriority@ lock 

 This should be done in a singleton goroutine that runs separately from the http request handlers, such that no more than 1 check-and-correct operation is in progress at any given time, and the http client doing a container update doesn't need to wait for it to finish. 

 When anything needs fixing, controller should log the timing and success/failure (according to rails) of each batch. 

Back