Feature #18863
[controller] add background job to clean up old container log records
0%
Description
As identified in https://dev.arvados.org/issues/18763#note-5, we have a "deleted_old_container_logs" rake task that is supposed to be running in a cron job to clear out old container logs.
Following the pattern we started using for the trash sweeps (#18339), add a background job that executes this sql query.
Remove the rake task file from the repository. Add a note to the upgrade nodes document that the cron job should be removed when upgrading.
The query used by the existing rake task is
DELETE FROM logs WHERE id in (SELECT logs.id FROM logs JOIN containers ON logs.object_uuid = containers.uuid WHERE event_type IN ('stdout', 'stderr', 'arv-mount', 'crunch-run', 'crunchstat') AND containers.log IS NOT NULL AND now() - containers.finished_at > interval '#{Rails.configuration.Containers.Logging.MaxAge.to_i} seconds')"
Determined empirically, a far more efficient version of that (if postgresql specific) is
delete from logs using containers where logs.object_uuid=containers.uuid and logs.event_type in ('stdout', 'stderr', 'arv-mount', 'crunch-run', 'crunchstat') AND containers.log IS NOT NULL AND now() - containers.finished_at > interval '#{Rails.configuration.Containers.Logging.MaxAge.to_i} seconds')"
Don't forget to remove the delete_old_container_logs.rake file from the API server after the new background job is implemented, cf. https://dev.arvados.org/issues/18763#note-5
Related issues
History
#1
Updated by Ward Vandewege 2 months ago
- Description updated (diff)
#2
Updated by Ward Vandewege 2 months ago
- Description updated (diff)
#3
Updated by Ward Vandewege 2 months ago
- Description updated (diff)
#4
Updated by Ward Vandewege 2 months ago
- Related to Bug #18763: remove unused rake tasks added
#5
Updated by Peter Amstutz 2 months ago
- Target version changed from 2022-03-30 Sprint to 2022-04-13 Sprint
#6
Updated by Peter Amstutz 2 months ago
- Related to Bug #18762: rails background tasks scaling issues added
#7
Updated by Peter Amstutz about 2 months ago
- Target version changed from 2022-04-13 Sprint to 2022-04-27 Sprint
#8
Updated by Peter Amstutz about 1 month ago
- Target version changed from 2022-04-27 Sprint to 2022-05-11 sprint
#9
Updated by Peter Amstutz 29 days ago
- Target version changed from 2022-05-11 sprint to 2022-05-25 sprint
#10
Updated by Peter Amstutz 19 days ago
- Target version changed from 2022-05-25 sprint to 2022-06-08 sprint
#11
Updated by Peter Amstutz 19 days ago
- Target version changed from 2022-06-08 sprint to 2022-06-22 Sprint
#12
Updated by Peter Amstutz 19 days ago
- Target version changed from 2022-06-22 Sprint to 2022-07-06