Bug #13593

[API] Sequence of "create group" requests runs slowly, and can crash API server

Added by Peter Amstutz 9 months ago. Updated 3 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
3.0

Description

A user reported that creating many projects at once resulted in extreme slowdown and eventually crashing the API server unless the create requests were throttled.

Creating a project (or any other group) also refreshes the permissions graph in the same transaction.

This makes the API call slow, but we intend/expect that other API calls can proceed while the update is in progress, so other (non-permission-modifying) API calls aren't expected to be slowed down.

The approach would be:

  • Accept an async param to permission-changing calls that would delay the permission graph update for a server configurable number of seconds in the future.
  • When updating the permission graph asynchronously, use the CONCURRENTLY parameter to avoid blocking SELECT while it’s being updated. (PostgreSQL 9.4 feature, check API server’s dependency requirements).
  • Use status 202 (Accepted) on async call responses.
  • Make a performance analysis comparison to make sure the issue is solved.
  • Update documentation.

Related issues

Blocks Arvados - Feature #13688: [API] optional asynchronous flag for create/update operationsNew

History

#1 Updated by Peter Amstutz 9 months ago

  • Status changed from New to In Progress

#2 Updated by Peter Amstutz 9 months ago

  • Description updated (diff)

#3 Updated by Tom Morris 9 months ago

  • Status changed from In Progress to New
  • Target version set to To Be Groomed

#4 Updated by Peter Amstutz 9 months ago

  • Subject changed from Batch create groups to Creating many groups is slow

#5 Updated by Peter Amstutz 9 months ago

  • Tracker changed from Feature to Bug
  • Description updated (diff)

#6 Updated by Tom Clegg 8 months ago

  • Subject changed from Creating many groups is slow to [API] Sequence of "create group" requests runs slowly, and can crash API server
  • Description updated (diff)

#7 Updated by Tom Clegg 8 months ago

  • Description updated (diff)

#8 Updated by Tom Clegg 8 months ago

  • Blocks Feature #13688: [API] optional asynchronous flag for create/update operations added

#9 Updated by Tom Clegg 8 months ago

  • Description updated (diff)

#10 Updated by Peter Amstutz 4 days ago

At Ward's suggestion, I'm bumping this since it's still an outstanding user request.

#11 Updated by Lucas Di Pentima 3 days ago

  • Description updated (diff)

#12 Updated by Lucas Di Pentima 3 days ago

  • Description updated (diff)
  • Story points set to 3.0

#13 Updated by Lucas Di Pentima 3 days ago

  • Target version changed from To Be Groomed to Arvados Future Sprints

Also available in: Atom PDF