Story #3149

[Workbench] Workbench infinite scroll and filter system should do filtering on server side instead of client side.

Added by Tom Clegg over 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Workbench
Target version:
Start date:
08/24/2014
Due date:
% Done:

100%

Estimated time:
(Total: 5.00 h)
Story points:
3.0

Description

Improvements we expect to see from this change:
  • Lower latency.
  • Better browser performance (thousands of hidden rows = bad news).
  • "Search" modal searches object contents, not just names and uuids. (The old /collections page uses the "any" filter to achieve this. Make sure there's an "any" filter usable with the arvados.v1.groups.contents method.)

Subtasks

Task #3671: Propagate search query to workbench when requesting rows in infinite scrollResolvedTom Clegg

Task #3672: Propagate search query from workbench to api serverResolvedTom Clegg

Task #3673: Clear results and refresh from server if search query is dirty and idle for 250msResolvedTom Clegg

Task #3674: Review 3149-filter-any (improve filter support in API)ResolvedTom Clegg

Task #3675: Review 3149-server-side-searchResolvedTom Clegg


Related issues

Related to Arvados - Bug #3692: [API] Eventbus filters parameter should use implicit AND (like REST filters) instead of implicit OR.Resolved10/10/2014

Associated revisions

Revision eba617ba
Added by Tom Clegg about 5 years ago

Merge branch '3149-filter-any' refs #3149

Revision e21b910a
Added by Tom Clegg about 5 years ago

Merge branch '3149-server-side-search' closes #3149

Revision 8cba6b1d (diff)
Added by Tom Clegg about 5 years ago

Fix test. refs #3149

History

#1 Updated by Tom Clegg over 5 years ago

  • Target version set to 2014-08-06 Sprint

#2 Updated by Radhika Chippada over 5 years ago

  • Subject changed from Workbench infinite scroll and filter system should do filtering on server side instead of client side. to [Workbench] Workbench infinite scroll and filter system should do filtering on server side instead of client side.
  • Category set to Workbench

#3 Updated by Tom Clegg over 5 years ago

  • Target version changed from 2014-08-06 Sprint to Arvados Future Sprints

#4 Updated by Tom Clegg about 5 years ago

  • Target version changed from Arvados Future Sprints to 2014-08-27 Sprint

#5 Updated by Tom Clegg about 5 years ago

  • Description updated (diff)

#6 Updated by Tom Clegg about 5 years ago

  • Assigned To set to Tom Clegg

#7 Updated by Tom Clegg about 5 years ago

  • Status changed from New to In Progress

#8 Updated by Peter Amstutz about 5 years ago

3149-filter-any:

  • Documentation update adds any as an example column but doesn't explain that it has a special meaning.
  • The behavior in eventbus.rb you asked me about this morning is a bug. It should join the list generated for each filter and returned in :cond_out using AND, and then join the resulting list of lists using OR.

The rest looks good.

#9 Updated by Peter Amstutz about 5 years ago

3149-server-side-search:

Usability notes about search. I realize not all of these (or even most of them) are directly related to server side search so we can decide which if any actually need to be addressed now.

  • When I click on the search button, I have to wait for it to load the modal, and then wait again for it to load the first page of results (this "cascaded AJAX loading" antipattern shows up when rendering tabs as well -- we should think about having the container elements of dialogs/tabs included with the initial page load, and use AJAX for the actual content load).
  • (bug) When I select a collection on the left hand side and it populates the preview panel preview, then change the search query so that item is no longer visible in the list, the preview panel stays populated with the old item until the I select something else.
  • I searched for "fastq" and got dozens of jobs and pipelines back as matches, but there's no preview content or other indication why they actually matched the search.
  • No way to narrow the search to just certain object types, e.g. I know I'm looking for a collection but I have to wade through dozens of hits on jobs before getting to the listing for collections.
  • It would be nice if the modal dialog was proportionally sized to the window the vertical dimension as well as horizontal.
  • It would be really nice if it told me how many elements are remaining next to the spinner, e.g. "loading next 20 elements out of 87 remaining". Sometimes the infinite scroll really feels like it goes on forever.
  • It should be more aggressive about loading the next page to hide latency. It should start loading the next page when scrolling reaches the 50% mark of the previous page (or maybe get within 2 window heights), not wait until the user gets all the way to the bottom before triggering.
Code:
  • Why are there two different naming styles "filterable-query-new" and "infiniteContentParamsFilterable"?
  • /^infiniteContentParams/ explain why there might be more several infiniteContentParams data elements? *

#10 Updated by Tom Clegg about 5 years ago

Peter Amstutz wrote:

3149-server-side-search:

Usability notes about search. I realize not all of these (or even most of them) are directly related to server side search so we can decide which if any actually need to be addressed now.

  • When I click on the search button, I have to wait for it to load the modal, and then wait again for it to load the first page of results (this "cascaded AJAX loading" antipattern shows up when rendering tabs as well -- we should think about having the container elements of dialogs/tabs included with the initial page load, and use AJAX for the actual content load).

Yes, that's definitely the way to go, and this branch takes a couple of steps in the right direction.

I noticed that although we do two round trips to Workbench, the net turnaround time is the same (either way it's dominated by Workbench loading and rendering the result rows) and offers a marginally better experience (you get a modal, instead of radio silence, while the results are loading).

  • (bug) When I select a collection on the left hand side and it populates the preview panel preview, then change the search query so that item is no longer visible in the list, the preview panel stays populated with the old item until the I select something else.
  • I searched for "fastq" and got dozens of jobs and pipelines back as matches, but there's no preview content or other indication why they actually matched the search.
  • No way to narrow the search to just certain object types, e.g. I know I'm looking for a collection but I have to wade through dozens of hits on jobs before getting to the listing for collections.
  • It would be nice if the modal dialog was proportionally sized to the window the vertical dimension as well as horizontal.
  • It would be really nice if it told me how many elements are remaining next to the spinner, e.g. "loading next 20 elements out of 87 remaining". Sometimes the infinite scroll really feels like it goes on forever.
  • It should be more aggressive about loading the next page to hide latency. It should start loading the next page when scrolling reaches the 50% mark of the previous page (or maybe get within 2 window heights), not wait until the user gets all the way to the bottom before triggering.

This is a good list of search todo's.

To clarify, this branch is just one search todo: move the filtering logic from the browser to the API server, so "search across all viewable objects" works without transmitting rendered versions of all viewable objects from server→workbench→browser.

Code:
  • Why are there two different naming styles "filterable-query-new" and "infiniteContentParamsFilterable"?

Fixed unnecessary usage of camelCase.

Added comment:
  •             // Note: We attach these data to DOM elements using
                // <element data-foo-bar="baz">. We store/retrieve them
                // using $('element').data('foo-bar'), although
                // .data('fooBar') would also work. The "all data" hash
                // returned by $('element').data(), however, always has
                // keys like 'fooBar'. In other words, where we have a
                // choice, we stick with the 'foo-bar' style to be
                // consistent with HTML. Here, our only option is
                // 'fooBar'.
    
  • /^infiniteContentParams/ explain why there might be more several infiniteContentParams data elements?
Updated comment:
  •         // Combine infiniteContentParams from multiple sources. This
            // mechanism allows each of several components to set and
            // update its own set of filters, without having to worry
            // about stomping on some other component's filters.
            //
            // For example, filterable.js writes filters in
            // infiniteContentParamsFilterable ("search for text foo")
            // without worrying about clobbering the filters set up by the
            // tab pane ("only show jobs and pipelines in this tab").
    

#11 Updated by Tom Clegg about 5 years ago

Peter Amstutz wrote:

3149-filter-any:

  • Documentation update adds any as an example column but doesn't explain that it has a special meaning.

Fixed.

  • The behavior in eventbus.rb you asked me about this morning is a bug. It should join the list generated for each filter and returned in :cond_out using AND, and then join the resulting list of lists using OR.

Added #3692

Thanks

#12 Updated by Anonymous about 5 years ago

  • Status changed from In Progress to Resolved

Applied in changeset arvados|commit:e21b910a307440a9d34e595673539a6e7d708e55.

Also available in: Atom PDF