Bug #21540
Updated by Peter Amstutz about 1 year ago
<pre> arvados.cwl-runner: [container example] error submitting container <HttpError 422 when requesting https://zzzzz.example.com/arvados/v1/container_requests/zzzzz-xvhdp-u047nzgnc3jdkd4?alt=json returned "//railsapi.internal/arvados/v1/container_requests/zzzzz-xvhdp-u047nzgnc3jdkd4: 422 Unprocessable Entity: #<ActiveRecord::Deadlocked: PG::TRDeadlockDetected: ERROR: deadlock detected DETAIL: Process 13792 waits for AccessExclusiveLock on tuple (801533,5) of relation 16562 of database 16400; blocked by process 31312. Process 31312 waits for ShareLock on transaction 508183640; blocked by process 19685. Process 19685 waits for ShareLock on transaction 508183565; blocked by process 13792. HINT: See server log for query details. : select 1 from containers where containers.uuid in ( select pri_container_uuid from container_tree($1) UNION select container_requests.requesting_container_uuid from container_requests where container_requests.container_uuid = $1 and container_requests.state = 'Committed' and container_requests.requesting_container_uuid is not NULL ) order by containers.uuid for update > (req-4d3u0781f6rol0q7xux1)"> </pre> This is almost certainly a lock ordering issue. We should: # Try to figure out what circumstances it happens why it's happening and if it is fixable/avoidable # Handle ActiveRecord::Deadlocked exceptions as 500 Internal Server Error so they are retried by the client