Story #4091

[Workbench] Add tests for undertested parts of Workbench, including infinite scroll.

Added by Tom Clegg about 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Radhika Chippada
Category:
-
Target version:
Start date:
10/27/2014
Due date:
% Done:

100%

Estimated time:
(Total: 2.00 h)
Story points:
1.0

Subtasks

Task #4326: Review branch: 4091-test-infinite-scrollingResolvedTom Clegg

Task #4328: Data collections tab for a project displaying all object types.ResolvedRadhika Chippada

Task #4329: Sort order created_at being ignored by the serverResolvedRadhika Chippada

Associated revisions

Revision 83fc1e7b (diff)
Added by Radhika Chippada about 5 years ago

refs #4091 - comment out the test that is failing intermittently in jenkins env.

Revision e2fe6c0e
Added by Radhika Chippada about 5 years ago

closes #4091
Merge branch '4091-test-infinite-scrolling'

History

#1 Updated by Tom Clegg about 5 years ago

  • Subject changed from [Workbench] Add infrastructure/patterns/examples to test untested parts of Workbench, like infinite scroll. to [Workbench] Add tests for undertested parts of Workbench, like infinite scroll.

#2 Updated by Ward Vandewege about 5 years ago

  • Target version changed from Arvados Future Sprints to 2014-10-29 sprint

#3 Updated by Tom Clegg about 5 years ago

  • Subject changed from [Workbench] Add tests for undertested parts of Workbench, like infinite scroll. to [Workbench] Add tests for undertested parts of Workbench, including infinite scroll.
  • Target version changed from 2014-10-29 sprint to Arvados Future Sprints

#4 Updated by Tom Clegg about 5 years ago

  • Target version changed from Arvados Future Sprints to 2014-10-29 sprint

#5 Updated by Radhika Chippada about 5 years ago

  • Assigned To set to Radhika Chippada

#6 Updated by Radhika Chippada about 5 years ago

  • Status changed from New to In Progress

#7 Updated by Radhika Chippada about 5 years ago

  • Status changed from In Progress to Resolved

Added tests for infinite scrolling: commit 3844c251d4db322759a23a6370faefb7c8e27728

#8 Updated by Radhika Chippada about 5 years ago

  • Status changed from Resolved to In Progress
  • Reopening it for the following reason:

The test with "2 pipelines and 200 jobs" is failing more than occasionally in jenkins env. Hence, we would like to use the "limit" on the project tabs, especially for the "Jobs and pipelines" tab

  • In addition, I observed a bug while testing with smaller than 200 limit

As of this writing, there are 103 objects in the Jobs_and_pipelines tab in the following project:
https://workbench.4xphq.arvadosapi.com/projects/4xphq-j7d0g-ur9v47pkvxu1hds

When I used a limit of 100 (to make things easy to see), I noticed that I am losing the 101st object (a job which has the same created_at as the previous page's last object).

We have this data loss problem with any limit size as long as the next page's item(s) have the same created_at time as the previous page's last item.

#9 Updated by Radhika Chippada about 5 years ago

Updated API server groups_controller to order on created_at time instead of uuid. This seems to have helped with the lost data. With this update, if approved, we should have all project tabs sorting on created_at time.

PS: I think sorting on modified_at time rather than created_at time more appropriate. But this may be for another day / another ticket.

#10 Updated by Tom Clegg about 5 years ago

At 97f3db9

I think the approach is correct here: if the sort key isn't always unique, get an overlapping page and skip the overlapping item(s). Suggestions:
  • Instead of adding or subtracting 1 second from the timestamp threshold, why not just replace the < and > nextpage_operators with <= and >=?
    • This did not work (I rechecked this morning also). Hence left it as is.
  • What if the last two items on page 1 have the same created_at timestamp? Doesn't that put us back in "too infinite" world? I think this can be addressed by adding a last-resort sort key ("@orders << 'uuid asc'"?) and rejecting the last page's last_uuid and everything before it from the subsequent page results. If an entire page has the same timestamp, that won't be good enough: infinite scroll will just stop. I think I could live with that for now, especially since it should be easy enough to detect and log/report. (Ultimately API server should offer a bomb-proof "next page" link, and we should just use that...)
    • I updated last_uuid to last_uuids array that keeps track of all the uuids in current page with the same created_at time as the last item and remove all of them from the next page objects. As you said, if all the objects in the next page have the same created_at time as the previous page, this will "end" scrolling. We could fix it, but I think this is simpler and probably desirable than making this code any more complex.
  • This might be a moot point given the preceding comment, but @objects.reject! do ... end would be more idiomatic than each+delete.
    • I like this. I updated the code accordingly.
For the per-tab page sizes: Instead of repeating the 200 in all of those views where we don't need to override the default page size, perhaps we could leave the default alone if not local_assigns[:limit], or even do something like this in app/views/projects/_show_tab_contents.html.erb?
- data-infinite-content-params-projecttab="<%= {limit: limit, filters: filters}.to_json %>">
+ data-infinite-content-params-projecttab="<%= local_assigns.to_json %>">
  • Fixed this

#11 Updated by Radhika Chippada about 5 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 67 to 100

Applied in changeset arvados|commit:e2fe6c0e5c1c62a37e03519590c04a5186a2cc9b.

Also available in: Atom PDF