Bug #8183
Updated by Tom Clegg about 8 years ago
A user created over 20,000 groups on one of our installations. For admin users (who can see every group) and for the user in question, Workbench appears to request all of these groups, 100 at a time, for each page load. An example from the api server logs: <pre> ... 10.28.0.7 - - [11/Jan/2016:19:33:58 +0000] "POST /arvados/v1/groups HTTP/1.1" 200 53511 "-" "HTTPClient/1.0 (2.6.0.1, ruby 2.1.4 (2014-10-27))" 10.28.0.7 - - [11/Jan/2016:19:33:59 +0000] "POST /arvados/v1/groups HTTP/1.1" 200 52721 "-" "HTTPClient/1.0 (2.6.0.1, ruby 2.1.4 (2014-10-27))" 10.28.0.7 - - [11/Jan/2016:19:33:59 +0000] "POST /arvados/v1/groups HTTP/1.1" 200 53510 "-" "HTTPClient/1.0 (2.6.0.1, ruby 2.1.4 (2014-10-27))" 10.28.0.7 - - [11/Jan/2016:19:34:00 +0000] "POST /arvados/v1/groups HTTP/1.1" 200 53510 "-" "HTTPClient/1.0 (2.6.0.1, ruby 2.1.4 (2014-10-27))" 10.28.0.7 - - [11/Jan/2016:19:34:00 +0000] "POST /arvados/v1/groups HTTP/1.1" 200 53508 "-" "HTTPClient/1.0 (2.6.0.1, ruby 2.1.4 (2014-10-27))" 10.28.0.7 - - [11/Jan/2016:19:34:00 +0000] "POST /arvados/v1/groups HTTP/1.1" 200 53507 "-" "HTTPClient/1.0 (2.6.0.1, ruby 2.1.4 (2014-10-27))" 10.28.0.7 - - [11/Jan/2016:19:34:00 +0000] "POST /arvados/v1/groups HTTP/1.1" 200 53512 "-" "HTTPClient/1.0 (2.6.0.1, ruby 2.1.4 (2014-10-27))" 10.28.0.7 - - [11/Jan/2016:19:34:01 +0000] "POST /arvados/v1/groups HTTP/1.1" 200 53511 "-" "HTTPClient/1.0 (2.6.0.1, ruby 2.1.4 (2014-10-27))" 10.28.0.7 - - [11/Jan/2016:19:34:01 +0000] "POST /arvados/v1/groups HTTP/1.1" 200 53506 "-" "HTTPClient/1.0 (2.6.0.1, ruby 2.1.4 (2014-10-27))" ... </pre> This is so slow that it causes the nginx reverse proxy to time out, making workbench unusable. The API server is fine - the cli interface works. h2. Proposed fix When building In the "my projects" tree for the workbench project dropdowns: * fetch one page dropdowns, instead of "all readable projects" (like we do now, but with "get all pages" turned off) * compare the returned result count to items_available * if items_available is > 3x returned result count (i.e., it would take more than 2 additional API calls to retrieve the full list), switch tactics and build the showing a tree of "my projects" tree this way instead: ** set my_project_uuids=[current_user.uuid] ** retrieve all projects", show only projects with owner_uuid in my_project_uuids (limit=200) ** add results to my_project_uuids ** repeat until no new results arrive, or my_project_uuids has more than 200 entries For now, = user's uuid, and show only the "choose project" dialog should use the old method, even though it will be slow when there are thousands first page of projects. We will come back and fix this part after #8286. results (ordered by name).