Today

08:28 PM Arvados Bug #13636: crunch-run takes a very long time for CWL steps with large numbers of File inputs - could use a new kind of mounts entry to address this

Update: actually, the amount of time it takes for crunch-run to prepare 58000 mounts appears to be quite a bit longer...

05:41 PM Arvados Bug #13636 (New): crunch-run takes a very long time for CWL steps with large numbers of File inputs - could use a new kind of mounts entry to address this

crunch-run appears to be able to process about 10 collection mounts per second (most of this time is spent performing...

06/15/2018

03:23 PM Arvados Bug #13630 (New): locking api_client_authorizations for update can deadlock ContainersController#update

We've observed an apparent deadlock (though not detected by postgres) when a containers update request (ContainersCon...

06/14/2018

10:50 AM Arvados Bug #11547: [API] Excessive memory and time consumed by API and dispatch when container fields (like "mounts") are large

Another example.
From passenger.log - I believe this was crunch attempting to update the container to say it has f...

06/13/2018

05:01 PM Arvados Bug #11547: [API] Excessive memory and time consumed by API and dispatch when container fields (like "mounts") are large

N.B. we have updated our fat passenger killer after discovering that advice regarding doing passenger instance gracef...

06/12/2018

02:13 PM Arvados Bug #11547: [API] Excessive memory and time consumed by API and dispatch when container fields (like "mounts") are large

We have both kinds. For example, our a-c-r runner containers have large workflow inputs that is mostly in cwl.input.j...

01:29 PM Arvados Bug #11547: [API] Excessive memory and time consumed by API and dispatch when container fields (like "mounts") are large

It looks like we are currently suffering this very same issue. Some of our container requests are very large, and our...

06/06/2018

09:29 AM Arvados Bug #13164: [API] dispatch sometimes tries to run cancelled containers at the expense of pending containers

Tom Clegg wrote:
> Some clarifications
> * "exclusive mode" doesn't block readers ("access exclusive mode" would do...

09:21 AM Arvados Bug #13164: [API] dispatch sometimes tries to run cancelled containers at the expense of pending containers

7e6ac5a4967614cbe59ed5c0ec41c8be4d4cff4d looks good to me in terms of fixing the issue with containers with nonzero p...

06/05/2018

04:09 PM Arvados Bug #13164: [API] dispatch sometimes tries to run cancelled containers at the expense of pending containers

Incidentally, the workaround query we were using above is overzealous - we are now running one which is more careful ...