Bug #10224

[FUSE] Fix expensive calls to log API

Added by Tom Morris almost 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
11/02/2016
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Story points:
-

Subtasks

Task #10437: Review 10224-efficient-event-poll-startupResolvedTom Clegg

History

#2 Updated by Tom Clegg almost 3 years ago

Copied from #10028:

I tried some queries on test servers and found
  • counting lots of rows in a huge table is slow, regardless of how great the index is: in order to count accurately, postgresql has to visit every counted row
  • given this, postgresql might be outsmarting us when it does a seq scan in our test trials. It knows most of the rows match the query, which means it will have to visit them anyway when it does a count -- so there is not much benefit in consulting the index first.
Typically, the huge table that causes trouble is "logs". For example:
  • arv-mount: subscribe to all events
           self.events = arvados.events.subscribe(self._api_client,
                                     [["event_type", "in", ["create", "update", "delete"]]],
                                     self.on_event)
    
  • Python SDK:
    items = self.api.logs().list(limit=1, order="id desc", filters=f).execute()['items']
  • API server:
         list[:items_available] = @objects.
            except(:limit).except(:offset).
            count(:id, distinct: true)
    
  • Postgresql: Slowly count all rows of the giant logs table with event_type ∈ {"create", "update", "delete"}.

Nobody here actually cares about items_available.

All PollClient needs is max(id) from the logs table, and only for the purpose of passing it to the next logs().list() call.

Perhaps it could pass a filter like [["created_at",">=",subscription_time]] (instead of doing this expensive API call to find the last known ID and then passing [["logs.id",">",str(self.id)]]) ...?

#3 Updated by Tom Clegg almost 3 years ago

10224-efficient-event-poll-startup

test 0e98d5f2f9c827deb5beb4a1765a4718d7cffd88

#4 Updated by Tom Clegg almost 3 years ago

  • Status changed from New to In Progress

#5 Updated by Tom Clegg almost 3 years ago

  • Target version set to 2016-11-09 sprint

#6 Updated by Tom Clegg almost 3 years ago

  • Assigned To set to Tom Clegg

#7 Updated by Radhika Chippada almost 3 years ago

0e98d5f

API unit tests failed (log_test)

One python test failed: test_subscribe_poll (tests.test_events.WebsocketTest)

I didn’t run the fuse tests

With the tests fixed, LGTM

#8 Updated by Tom Clegg almost 3 years ago

API unit tests fixed.

Python SDK tests pass for me. We'll see what Jenkins thinks -- build 56 queued at https://ci.curoverse.com/job/developer-run-tests/

#9 Updated by Tom Clegg almost 3 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF