Arvados

• Fixed scrolling issue. The issue was in any tab that is displaying multiple data kinds. In this case, the first page was being rendered repeatedly instead of rending the next page

• Tested the issue resolution manually. Verified that the "next" page content is rendered. And the scrolling stops when all the items are rendered.

• Added "multiple objects" as test fixtures. Updated workbench test_helper to be able to handle the multiple object definitions in erb for loop.

• Added tests to load page with large data sets and scroll to the bottom of the page. The "second page" does not load in the test. I think this is because the page display is not updated unless it is by a click event.

• The test "scroll pipeline instances tab . . ." for "project with 250 pipelines" is failing when I try to use the size 250 or even 50, but passes on lower amounts such as 25. I am thinking it is failing because too much data is being sent from server in the test env. Or, not sure if it is actually indicating a real issue.
  • If it is due to size, I think we should use a config parameter for "@limit" in workbench application_controller. Currently it is hard coded at 200 for all paging requirement. I think it is desirable that we use 200 for "projects list drop-down" and a config parameter of a suitable size of actual objects (such as pipeline instances, collections etc).

Radhika Chippada wrote:

• Fixed scrolling issue. The issue was in any tab that is displaying multiple data kinds. In this case, the first page was being rendered repeatedly instead of rending the next page

• Tested the issue resolution manually. Verified that the "next" page content is rendered. And the scrolling stops when all the items are rendered.

Awesome, thanks.

• Added "multiple objects" as test fixtures. Updated workbench test_helper to be able to handle the multiple object definitions in erb for loop.

I can live with this for the sake of testing infinite scroll, at least until we set up some factories & database cleaner infrastructure (or provide some other access to the fixtures other than peeking into the yml file from other languages/projects).

Is there a reason why Workbench needs to omit those fixtures rather than parsing the YML file as an erb template just like apiserver does? (Obviously Python won't be able to do erb, but Workbench could...)

• Added tests to load page with large data sets and scroll to the bottom of the page. The "second page" does not load in the test. I think this is because the page display is not updated unless it is by a click event.

Hm, it should be triggered by a "scroll" event. I'll see if I can spot anything. Otherwise maybe we should back away slowly for now, and come back to it later when its guard is down?

• The test "scroll pipeline instances tab . . ." for "project with 250 pipelines" is failing when I try to use the size 250 or even 50, but passes on lower amounts such as 25. I am thinking it is failing because too much data is being sent from server in the test env. Or, not sure if it is actually indicating a real issue.
  • If it is due to size, I think we should use a config parameter for "@limit" in workbench application_controller. Currently it is hard coded at 200 for all paging requirement. I think it is desirable that we use 200 for "projects list drop-down" and a config parameter of a suitable size of actual objects (such as pipeline instances, collections etc).

I suspect it's timing out just because pipeline instances are very slow to render. (Users also take issue with this!)

Reducing the default to 50 (?) would be reasonable. But it should also be possible to reduce the limit further for that particular case rather than application-wide by passing a limit=20 parameter -- perhaps in _show_tab_contents.html.erb along with {filters: filters}?

(200 is just the default if no limit param is given -- any request should be able to override it.)

Tom Clegg wrote:

Hm, it should be triggered by a "scroll" event. I'll see if I can spot anything.

(Copied from IRC) If I change scrollBy(0,10000) to scrollBy(0,999000) and eliminate the scrollbar_present test, I get 201 rows:

-        scrollbar_present = page.execute_script("return document.documentElement.scrollHeight>document.documentElement.clientHeight;");
-        if scrollbar_present
-          page.execute_script "window.scrollBy(0,10000)" 
-          begin
-            wait_for_ajax
-          rescue
-          end
-        end
+        page.execute_script "window.scrollBy(0,999000)" 
+        begin
+          wait_for_ajax
+        rescue
+        end

• Tom, thanks for the tip about scrolling size. Infinite scrolling is now working in the test

• I noticed that the created_at filter was not being sent to the API server. I added it to filters and now the second page retrieval and scrolling only as needed are working. Let me know if you think of a better way of preserving the created_at filter or why it is getting lost in the first place. Otherwise, I will stick with what I have.

• You are correct about the timing issue. I am now using less number of pipelines and more number of jobs to push the page size beyond 200

• I have not implemented limit size of 20 for "Jobs and pipelines" tab. I think it will require it's own set of tests and better to do as a separate issue.

Radhika Chippada wrote:

• Tom, thanks for the tip about scrolling size. Infinite scrolling is now working in the test

YW!

• I noticed that the created_at filter was not being sent to the API server. I added it to filters and now the second page retrieval and scrolling only as needed are working. Let me know if you think of a better way of preserving the created_at filter or why it is getting lost in the first place. Otherwise, I will stick with what I have.

I think the problem is just that infinite_scroll.js tries to add filters to the "next page" URI: It should just believe that the URI provided by Rails is correct.

Have a look at branch 4062-infinite-scroll-repeat-TC (1f168b8) -- does this seem right? (I branched from an earlier point on your branch, before the created_at_filter change.)

• You are correct about the timing issue. I am now using less number of pipelines and more number of jobs to push the page size beyond 200

• I have not implemented limit size of 20 for "Jobs and pipelines" tab. I think it will require it's own set of tests and better to do as a separate issue.

SGTM.