Bug #14507
Updated by Peter Amstutz about 6 years ago
(update) Seems to be a configuration problem #14508 CWL jobs are failing with this: > ApiError: <HttpError 403 when requesting https://4xphq.arvadosapi.com/arvados/v1/container_requests?alt=json returned "database connection error"> Here's the relevant log: > Nov 19 14:48:18 4xphq.arvadosapi.com arvados-controller[31483]: {"RequestID":"req-i7f0fqu8xkbbf313sh9r","level":"info","msg":"request","remoteAddr":"127.0.0.1:14874","reqBytes":2770, > "reqForwardedFor":"144.121.5.198","reqHost":"4xphq.arvadosapi.com","reqMethod":"POST","reqPath":"arvados/v1/container_requests","reqQuery":"alt=json","time":"2018-11-19T14:48:18.229777247Z"} > Nov 19 14:48:18 4xphq.arvadosapi.com arvados-controller[31483]: {"RequestID":"req-i7f0fqu8xkbbf313sh9r", > "error":"pq: password authentication failed for user \"root\"","level":"error", > "msg":"postgresql connect succeeded but ping failed","remoteAddr":"127.0.0.1:14874","reqBytes":2770,"reqForwardedFor":"144.121.5.198", > "reqHost":"4xphq.arvadosapi.com","reqMethod":"POST","reqPath":"arvados/v1/container_requests","reqQuery":"alt=json","time":"2018-11-19T14:48:18.413792406Z"} > Nov 19 14:48:18 4xphq.arvadosapi.com arvados-controller[31483]: {"RequestID":"req-i7f0fqu8xkbbf313sh9r","level":"info","msg":"response","remoteAddr":"127.0.0.1:14874", > "reqBytes":2770,"reqForwardedFor":"144.121.5.198","reqHost":"4xphq.arvadosapi.com","reqMethod":"POST","reqPath":"arvados/v1/container_requests","reqQuery":"alt=json","respBytes":41, > "respStatus":"Forbidden","respStatusCode":403,"time":"2018-11-19T14:48:18.413870566Z","timeToStatus":0.184078,"timeTotal":0.184089,"timeWriteBody":0.000012} The operative error messages seem to be @pq: password authentication failed for user \"root\"@ and @postgresql connect succeeded but ping failed@ Perhaps connections in the db connection pool are going stale?