Project

General

Profile

Actions

Bug #21547

open

return certain database errors as 500 so they can be retried

Added by Peter Amstutz 7 months ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Assigned To:
Category:
API
Target version:
Story points:
-

Description

Certain database errors represent transient errors. We should tell the client to retry the request by returning a 500 internal server error instead of 422 (which is the default behavior).

#<ActiveRecord::Deadlocked: PG::TRDeadlockDetected: ERROR: deadlock detected>

Rationale: The observed deadlocks in Arvados are conflicts between two statements (a lock ordering issue), so unwinding and retrying is a reasonable solution

#<ActiveRecord::StatementInvalid: PG::UnableToSend>

Rationale: It seems this gets thrown when the API server can't connect to the database.

Here's the list of postgres errors known to the PG gem:

https://github.com/ged/ruby-pg/blob/daec80f91b9519509ca1694a231f11a75cb43f7f/ext/errorcodes.def#L598

https://github.com/ged/ruby-pg/blob/daec80f91b9519509ca1694a231f11a75cb43f7f/ext/pg_errors.c#L88

Some other possible Exceptions to retry:

ConnectionBad
ConnectionException
ConnectionDoesNotExist
ConnectionFailure
TooManyConnections
CannotConnectNow
IdleSessionTimeout
ObjectInUse
LockNotAvailable
AdminShutdown
CrashShutdown

(There's a lot of connection related errors and I don't know the difference between them, but I included them all because it seems like those are very likely to be errors that occur through no fault of the client).


Subtasks 1 (1 open0 closed)

Task #21554: ReviewNewTom CleggActions

Related issues

Related to Arvados - Bug #21540: occasional container_requests deadlockNewActions
Actions #1

Updated by Peter Amstutz 7 months ago

  • Related to Bug #21540: occasional container_requests deadlock added
Actions #2

Updated by Peter Amstutz 7 months ago

  • Description updated (diff)
Actions #3

Updated by Peter Amstutz 7 months ago

  • Description updated (diff)
Actions #4

Updated by Peter Amstutz 7 months ago

  • Assigned To set to Peter Amstutz
Actions #5

Updated by Peter Amstutz 7 months ago

  • Target version changed from Development 2024-03-13 sprint to Development 2024-03-27 sprint
Actions #6

Updated by Peter Amstutz 6 months ago

  • Target version changed from Development 2024-03-27 sprint to Development 2024-04-10 sprint
Actions #7

Updated by Peter Amstutz 6 months ago

  • Target version changed from Development 2024-04-10 sprint to Development 2024-04-24 sprint
Actions #8

Updated by Peter Amstutz 6 months ago

  • Target version changed from Development 2024-04-24 sprint to Development 2024-05-08 sprint
Actions #9

Updated by Peter Amstutz 6 months ago

  • Target version changed from Development 2024-05-08 sprint to Development 2024-05-22 sprint
Actions #10

Updated by Peter Amstutz 5 months ago

  • Target version changed from Development 2024-05-22 sprint to Development 2024-06-05 sprint
Actions #11

Updated by Peter Amstutz 5 months ago

  • Target version changed from Development 2024-06-05 sprint to Future
Actions

Also available in: Atom PDF