Bug #21904
closedRemove feature that rejects POST arvados/v1/logs immediately when any requests are queued
Description
In #20602 we started rejecting "POST /arvados/v1/logs" requests instead of queueing them when all active request slots are full. The idea was to drop container-logging events outright during busy periods (i.e., when any requests are queued) -- otherwise, when lots of containers were running, these frequent requests would quickly fill up the queue and cause more important non-interactive requests, like saving container output collections, to fail.
This is no longer needed because crunch-run no longer sends POST /arvados/v1/logs
requests at all.
Now, POST /arvados/v1/logs
requests are more likely to come from the "log uploads and downloads" feature of keep-web, and the above rationale does not apply.
Instead, these should be queued the same way as other non-interactive requests.
(This has also caused an additional problem where keep-web's outgoing request throttle gets triggered by the premature 503s. That problem could be fixed on the client side, but a client-side fix wouldn't address the problem of upload/download logs being skipped whenever the controller is even moderately busy. So it makes more sense to just fix the server side.)
Updated by Tom Clegg 6 months ago
- Related to Bug #21748: awscli downloads from keep-web slowly added
Updated by Peter Amstutz 6 months ago
- Target version set to Development 2024-08-07 sprint
Updated by Tom Clegg 5 months ago
21904-no-unqueueable-reqs @ df73561dc9cc547df259d5215a482cd7d787aed6 -- developer-run-tests: #4352
Updated by Brett Smith 5 months ago
Tom Clegg wrote in #note-4:
21904-no-unqueueable-reqs @ df73561dc9cc547df259d5215a482cd7d787aed6 -- developer-run-tests: #4352
Looks good to me. grepped for a few different patterns to look for remnant code and didn't find anything relevant. Thanks.
Updated by Tom Clegg 5 months ago
- Status changed from In Progress to Resolved
Applied in changeset arvados|f9ed354eb7b6940df85833c566128487dd29d806.