Bug #21540
Updated by Peter Amstutz 3 months ago
<pre>
arvados.cwl-runner: [container example] error submitting container
<HttpError 422 when requesting https://zzzzz.example.com/arvados/v1/container_requests/zzzzz-xvhdp-u047nzgnc3jdkd4?alt=json returned
"//railsapi.internal/arvados/v1/container_requests/zzzzz-xvhdp-u047nzgnc3jdkd4: 422 Unprocessable Entity: #<ActiveRecord::Deadlocked: PG::TRDeadlockDetected: ERROR: deadlock detected
DETAIL: Process 13792 waits for AccessExclusiveLock on tuple (801533,5) of relation 16562 of database 16400; blocked by process 31312.
Process 31312 waits for ShareLock on transaction 508183640; blocked by process 19685.
Process 19685 waits for ShareLock on transaction 508183565; blocked by process 13792.
HINT: See server log for query details. : select 1 from containers where containers.uuid in ( select pri_container_uuid from container_tree($1) UNION select container_requests.requesting_container_uuid from container_requests where container_requests.container_uuid = $1 and container_requests.state = 'Committed' and container_requests.requesting_container_uuid is not NULL ) order by containers.uuid for update > (req-4d3u0781f6rol0q7xux1)">
</pre>
This is almost certainly a lock ordering issue. We should:
# Try to figure out why it's happening and if it is fixable/avoidable
# Handle ActiveRecord::Deadlocked exceptions as 500 Internal Server Error so they are retried by the client