Bug #4062

[Workbench] Infinite scrolling in the 'Jobs and pipelines' tab should not keep repeating the same list over and over again

Added by Ward Vandewege about 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Radhika Chippada
Category:
-
Target version:
Start date:
10/16/2014
Due date:
% Done:

100%

Estimated time:
(Total: 1.00 h)
Story points:
0.5

Description

It's a little bit too infinite...

Scroll to the bottom of the list, magic loader kicks in, and loads the same list again at the bottom of the page. And you can keep doing this.


Subtasks

Task #4226: Review 4062-infinite-scroll-repeat-issue (this addresses 4062 and 4091)ResolvedRadhika Chippada

Associated revisions

Revision 3844c251
Added by Radhika Chippada about 5 years ago

closes #4062
Merge branch '4062-infinite-scroll-repeat-TC'

History

#1 Updated by Ward Vandewege about 5 years ago

  • Description updated (diff)

#2 Updated by Ward Vandewege about 5 years ago

  • Description updated (diff)

#3 Updated by Tom Clegg about 5 years ago

  • Subject changed from [Workbench] infinite scrolling in the 'Jobs and pipelines' tab just keeps repeating the same list over and over again to [Workbench] Infinite scrolling in the 'Jobs and pipelines' tab should not keep repeating the same list over and over again

#4 Updated by Tom Clegg about 5 years ago

  • Target version changed from Bug Triage to Arvados Future Sprints

#5 Updated by Ward Vandewege about 5 years ago

  • Target version changed from Arvados Future Sprints to 2014-10-29 sprint

#6 Updated by Radhika Chippada about 5 years ago

  • Assigned To set to Radhika Chippada

#7 Updated by Radhika Chippada about 5 years ago

  • Status changed from New to In Progress

#8 Updated by Radhika Chippada about 5 years ago

  • Fixed scrolling issue. The issue was in any tab that is displaying multiple data kinds. In this case, the first page was being rendered repeatedly instead of rending the next page
  • Tested the issue resolution manually. Verified that the "next" page content is rendered. And the scrolling stops when all the items are rendered.
  • Added "multiple objects" as test fixtures. Updated workbench test_helper to be able to handle the multiple object definitions in erb for loop.
  • Added tests to load page with large data sets and scroll to the bottom of the page. The "second page" does not load in the test. I think this is because the page display is not updated unless it is by a click event.
  • The test "scroll pipeline instances tab . . ." for "project with 250 pipelines" is failing when I try to use the size 250 or even 50, but passes on lower amounts such as 25. I am thinking it is failing because too much data is being sent from server in the test env. Or, not sure if it is actually indicating a real issue.
    • If it is due to size, I think we should use a config parameter for "@limit" in workbench application_controller. Currently it is hard coded at 200 for all paging requirement. I think it is desirable that we use 200 for "projects list drop-down" and a config parameter of a suitable size of actual objects (such as pipeline instances, collections etc).

#9 Updated by Tom Clegg about 5 years ago

Radhika Chippada wrote:

  • Fixed scrolling issue. The issue was in any tab that is displaying multiple data kinds. In this case, the first page was being rendered repeatedly instead of rending the next page
  • Tested the issue resolution manually. Verified that the "next" page content is rendered. And the scrolling stops when all the items are rendered.

Awesome, thanks.

  • Added "multiple objects" as test fixtures. Updated workbench test_helper to be able to handle the multiple object definitions in erb for loop.

I can live with this for the sake of testing infinite scroll, at least until we set up some factories & database cleaner infrastructure (or provide some other access to the fixtures other than peeking into the yml file from other languages/projects).

Is there a reason why Workbench needs to omit those fixtures rather than parsing the YML file as an erb template just like apiserver does? (Obviously Python won't be able to do erb, but Workbench could...)

  • Added tests to load page with large data sets and scroll to the bottom of the page. The "second page" does not load in the test. I think this is because the page display is not updated unless it is by a click event.

Hm, it should be triggered by a "scroll" event. I'll see if I can spot anything. Otherwise maybe we should back away slowly for now, and come back to it later when its guard is down?

  • The test "scroll pipeline instances tab . . ." for "project with 250 pipelines" is failing when I try to use the size 250 or even 50, but passes on lower amounts such as 25. I am thinking it is failing because too much data is being sent from server in the test env. Or, not sure if it is actually indicating a real issue.
    • If it is due to size, I think we should use a config parameter for "@limit" in workbench application_controller. Currently it is hard coded at 200 for all paging requirement. I think it is desirable that we use 200 for "projects list drop-down" and a config parameter of a suitable size of actual objects (such as pipeline instances, collections etc).

I suspect it's timing out just because pipeline instances are very slow to render. (Users also take issue with this!)

Reducing the default to 50 (?) would be reasonable. But it should also be possible to reduce the limit further for that particular case rather than application-wide by passing a limit=20 parameter -- perhaps in _show_tab_contents.html.erb along with {filters: filters}?

(200 is just the default if no limit param is given -- any request should be able to override it.)

#10 Updated by Tom Clegg about 5 years ago

Tom Clegg wrote:

Hm, it should be triggered by a "scroll" event. I'll see if I can spot anything.

(Copied from IRC) If I change scrollBy(0,10000) to scrollBy(0,999000) and eliminate the scrollbar_present test, I get 201 rows:

-        scrollbar_present = page.execute_script("return document.documentElement.scrollHeight>document.documentElement.clientHeight;");
-        if scrollbar_present
-          page.execute_script "window.scrollBy(0,10000)" 
-          begin
-            wait_for_ajax
-          rescue
-          end
-        end
+        page.execute_script "window.scrollBy(0,999000)" 
+        begin
+          wait_for_ajax
+        rescue
+        end

#11 Updated by Radhika Chippada about 5 years ago

  • Tom, thanks for the tip about scrolling size. Infinite scrolling is now working in the test
  • I noticed that the created_at filter was not being sent to the API server. I added it to filters and now the second page retrieval and scrolling only as needed are working. Let me know if you think of a better way of preserving the created_at filter or why it is getting lost in the first place. Otherwise, I will stick with what I have.
  • You are correct about the timing issue. I am now using less number of pipelines and more number of jobs to push the page size beyond 200
  • I have not implemented limit size of 20 for "Jobs and pipelines" tab. I think it will require it's own set of tests and better to do as a separate issue.

#12 Updated by Tom Clegg about 5 years ago

Radhika Chippada wrote:

  • Tom, thanks for the tip about scrolling size. Infinite scrolling is now working in the test

YW!

  • I noticed that the created_at filter was not being sent to the API server. I added it to filters and now the second page retrieval and scrolling only as needed are working. Let me know if you think of a better way of preserving the created_at filter or why it is getting lost in the first place. Otherwise, I will stick with what I have.

I think the problem is just that infinite_scroll.js tries to add filters to the "next page" URI: It should just believe that the URI provided by Rails is correct.

Have a look at branch 4062-infinite-scroll-repeat-TC (1f168b8) -- does this seem right? (I branched from an earlier point on your branch, before the created_at_filter change.)

  • You are correct about the timing issue. I am now using less number of pipelines and more number of jobs to push the page size beyond 200
  • I have not implemented limit size of 20 for "Jobs and pipelines" tab. I think it will require it's own set of tests and better to do as a separate issue.

SGTM.

#13 Updated by Radhika Chippada about 5 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:3844c251d4db322759a23a6370faefb7c8e27728.

Also available in: Atom PDF