Feature #20220
Updated by Peter Amstutz about 1 year ago
https://dev.arvados.org/projects/arvados/wiki/Efficient_live_access_to_container_logs
Dispatcher should periodically contact each running container and fetch the sizes of the live logs. For all the logs that have new data, fetch the data and write them to files local to the dispatcher.
When the container is about to go into a cancelled or complete state, compare the container log collection to the local files. If the local files have content that isn't in the log collection, update the log collection. Then local logs can be discarded.