[API] [Workbench] Delete old container log messages once saved in Keep
Extend source:services/api/lib/tasks/delete_old_job_logs.rake so
rake db:delete_old_job_logs deletes container log entries once they're saved in Keep (currently it only deletes job log entries with event_type="stderr").
In the Workbench container log viewer, detect the case where log entries have been deleted from the logs table, and fetch them from Keep instead.
If fetching them from Keep doesn't work (or is not implemented), rather than showing a mysteriously empty log viewer, show a message acknowledging that the logs aren't shown ("because the container is finished and its logs have been archived"?).
#6 Updated by Radhika Chippada over 4 years ago
Branch 9514-delete-old-container-logs ready for review at 509f028
- Added a separate delete_old_container_logs task and tests
- Moved job related code from log_viewer.js into a separate job_log_viewer.js. Moved previous addToLogViewer code into job_log_viewer and renamed as addToJobLogViewer and converted addToLogViewer to handle generic message display
- Added log-viewer div and code similar to job log viewer into views/work_units/_show_log.html.erb
#7 Updated by Peter Amstutz over 4 years ago
- (as mentioned in chat) seems like it would be simpler and more efficient to issue a single delete statement, instead of deleting in batches. I checked the log and this is producing statements like:
DELETE FROM "logs" WHERE "logs"."id" IN (139, 141, 142, ... (1000s of ids) ... )
- I'm not sure the log viewer loading the contents of crunch-run.txt is providing any value. Rather than creeping scope with a bunch of suggestions, maybe we should just disable it for crunch v2 logs until we can give it the attention it deserves. (We can keep the other refactoring.)
#8 Updated by Radhika Chippada over 4 years ago
"seems like it would be simpler and more efficient to issue a single delete statement ..." :
I was able to do this with pure SQL with using ActiveRecord syntax. Example SQL statement from API server log when the test runs:
DELETE FROM logs WHERE id in (SELECT logs.id FROM logs JOIN containers ON logs.object_uuid = containers.uuid WHERE event_type IN ('stdout', 'stderr', 'arv-mount', 'crunch-run', 'crunchstat') AND containers.log IS NOT NULL AND containers.finished_at < '2015-09-30 23:34:14 UTC')
"I'm not sure the log viewer loading the contents of crunch-run.txt is providing any value ..." :
In this new branch, only deleting old logs. Updated the #Log for containers to not show the log div in terminal when it is empty
#11 Updated by Radhika Chippada over 4 years ago
Updated the delete_old_job_logs task also to use this better performing SQL. The log statement from API server when tests are run:
DELETE FROM logs WHERE id in (SELECT logs.id FROM logs JOIN jobs ON logs.object_uuid = jobs.uuid WHERE event_type = 'stderr' AND jobs.log IS NOT NULL AND jobs.finished_at < '2015-10-02 14:14:04 UTC')