Bug #21904
openRemove feature that rejects POST arvados/v1/logs immediately when any requests are queued
Description
In #20602 we started rejecting "POST /arvados/v1/logs" requests instead of queueing them when all active request slots are full. The idea was to drop container-logging events outright during busy periods (i.e., when any requests are queued) -- otherwise, when lots of containers were running, these frequent requests would quickly fill up the queue and cause more important non-interactive requests, like saving container output collections, to fail.
This is no longer needed because crunch-run no longer sends POST /arvados/v1/logs
requests at all.
Now, POST /arvados/v1/logs
requests are more likely to come from the "log uploads and downloads" feature of keep-web, and the above rationale does not apply.
Instead, these should be queued the same way as other non-interactive requests.
(This has also caused an additional problem where keep-web's outgoing request throttle gets triggered by the premature 503s. That problem could be fixed on the client side, but a client-side fix wouldn't address the problem of upload/download logs being skipped whenever the controller is even moderately busy. So it makes more sense to just fix the server side.)
Related issues
Updated by Tom Clegg 15 days ago
- Related to Bug #21748: awscli downloads from keep-web slowly? added
Updated by Peter Amstutz 1 day ago
- Target version set to Development 2024-08-07 sprint