Bug #6074
Updated by Ward Vandewege almost 9 years ago
<pre>
On Mon, May 18, 2015 at 06:51:02PM -0700, Misha Zatsman wrote:
> I've been working on the pull lists lately, using stale data because the
> api server has been frustratingly slow to respond to the data manager. Here
> you can see it taking about a minute plus to respond, before eventually
> failing:
>
> Is there a fix in the works? If the API server takes one second per
> collection to respond, the data manager will never be able to run in a
> reasonable amount of time.
>
> This all happened around 6:30 PM san francisco time in case you want to
> look at the qr1hi api server logs.
>
> 2015/05/19 01:23:41 15570 collections read, 49 new in last batch,
> 2014-10-28T15:36:52Z latest modified date, 16113 8991033 250878093
> avg,max,total manifest size
> 2015/05/19 01:24:36 15619 collections read, 49 new in last batch,
> 2014-10-29T11:50:18Z latest modified date, 16064 8991033 250909919
> avg,max,total manifest size
> 2015/05/19 01:25:36 15667 collections read, 48 new in last batch,
> 2014-10-30T00:46:12Z latest modified date, 16018 8991033 250948820
> avg,max,total manifest size
> 2015/05/19 01:26:36 15716 collections read, 49 new in last batch,
> 2014-10-30T19:30:56Z latest modified date, 15976 8991033 251073129
> avg,max,total manifest size
> 2015/05/19 01:27:43 15765 collections read, 49 new in last batch,
> 2014-10-31T23:45:52Z latest modified date, 15927 8991033 251083331
> avg,max,total manifest size
> 2015/05/19 01:28:58 15814 collections read, 49 new in last batch,
> 2014-11-02T23:45:31Z latest modified date, 15879 8991033 251116752
> avg,max,total manifest size
> 2015/05/19 01:30:19 15863 collections read, 49 new in last batch,
> 2014-11-04T11:29:52Z latest modified date, 15840 8991033 251277070
> avg,max,total manifest size
> 2015/05/19 01:31:33 15912 collections read, 49 new in last batch,
> 2014-11-04T16:25:36Z latest modified date, 15798 8991033 251385001
> avg,max,total manifest size
> 2015/05/19 01:32:47 15961 collections read, 49 new in last batch,
> 2014-11-05T04:30:20Z latest modified date, 15751 8991033 251406912
> avg,max,total manifest size
> 2015/05/19 01:33:58 16010 collections read, 49 new in last batch,
> 2014-11-05T21:50:34Z latest modified date, 15705 8991033 251430464
> avg,max,total manifest size
> 2015/05/19 01:35:21 16059 collections read, 49 new in last batch,
> 2014-11-06T16:12:59Z latest modified date, 15658 8991033 251456615
> avg,max,total manifest size
> 2015/05/19 01:37:35 16108 collections read, 49 new in last batch,
> 2014-11-06T23:46:04Z latest modified date, 45871 54080537 738893606
> avg,max,total manifest size
> 2015/05/19 01:38:46 16157 collections read, 49 new in last batch,
> 2014-11-07T20:08:25Z latest modified date, 55791 54080537 901420414
> avg,max,total manifest size
> 2015/05/19 01:39:32 16206 collections read, 49 new in last batch,
> 2014-11-07T22:52:48Z latest modified date, 55637 54080537 901649169
> avg,max,total manifest size
> 2015/05/19 01:40:27 16255 collections read, 49 new in last batch,
> 2014-11-08T07:17:22Z latest modified date, 55473 54080537 901715163
> avg,max,total manifest size
> 2015/05/19 01:41:09 16303 collections read, 48 new in last batch,
> 2014-11-08T21:04:18Z latest modified date, 55314 54080537 901778597
> avg,max,total manifest size
> 2015/05/19 01:41:53 16352 collections read, 49 new in last batch,
> 2014-11-09T17:48:07Z latest modified date, 55149 54080537 901804169
> avg,max,total manifest size
> 2015/05/19 01:43:01 16401 collections read, 49 new in last batch,
> 2014-11-10T18:26:31Z latest modified date, 65035 54085104 1066634271
> avg,max,total manifest size
> 2015/05/19 01:44:01 Error querying collections: API server (
> qr1hi.arvadosapi.com) returned 422: 422 Unprocessable Entity. Additional
> details: #<NoMemoryError: failed to allocate memory>
> exit status 1
>
</pre>
Also - qr1hi is currently running commit:31d8ece. That is after #5834 was merged. And still there is a NoMemoryError.