Project

General

Profile

Actions

Bug #20424

closed

Process view page slowdown when inputs panel is ON and workflow with several thousands inputs

Added by Lucas Di Pentima 11 months ago. Updated 11 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Workbench2
Story points:
2.0
Release relationship:
Auto

Description

Customer reported having significant slowdown on the process view page when trying to see a process that has ~4300 inputs. When closing the inputs panel & clicking on the reload button, the page renders a lot quicker.


Files

lots-of-inputs.json (1.15 MB) lots-of-inputs.json Lucas Di Pentima, 04/27/2023 07:40 PM
lots-of-inputs.json (1.15 MB) lots-of-inputs.json Peter Amstutz, 05/10/2023 06:52 PM

Subtasks 1 (0 open1 closed)

Task #20439: ReviewResolvedPeter Amstutz05/10/2023Actions

Related issues

Related to Arvados - Bug #21508: Browser struggles with very large number of input or output parametersIn ProgressStephen SmithActions
Actions #1

Updated by Peter Amstutz 11 months ago

Oh, that's quite a lot. It should handle it better.

I did notice when I was reusing the component for the registered workflow panel that it didn't seem to be using React "key" in the list components to reduce re-renders, I thought I added it, but maybe only in one place.

Actions #2

Updated by Lucas Di Pentima 11 months ago

  • File many inputs slowdown.mp4 added
Actions #3

Updated by Lucas Di Pentima 11 months ago

  • File deleted (many inputs slowdown.mp4)
Actions #4

Updated by Peter Amstutz 11 months ago

  • Target version changed from Development 2023-04-26 sprint to Development 2023-05-10 sprint
Actions #6

Updated by Peter Amstutz 11 months ago

  • Story points set to 2.0
Actions #7

Updated by Peter Amstutz 11 months ago

  • Assigned To set to Stephen Smith
Actions #8

Updated by Lucas Di Pentima 11 months ago

Added an example of CR with lots of inputs.

Actions #9

Updated by Stephen Smith 11 months ago

  • Status changed from New to In Progress

Changes at arvados-workbench2|3e18e35172808479d02e02985563d658fec7bb0f branch 20424-io-panel-performance

  • Memoized the process io preview component, reduces re-renders from ~140 to 6
  • Rewrote file array processing code to use a single loop, cuts processing time in half (~5s to ~2s) * The remaining performance impact here seems to be solely due to having to run checks on all 4000 entries, each execution is <1ms

The remaining performance issues seem to stem mostly from the rendering code on process-io-card.tsx L400 and having to render 4000 entries. I'm not sure how to further optimize that part

Actions #10

Updated by Lucas Di Pentima 11 months ago

If the issue is rendering so many elements, we have solved that in the past by using <FixedSizeList> and <AutoSizer> to implement virtualized lists that only render the elements that are on the current viewport.

Would that help?

Actions #11

Updated by Peter Amstutz 11 months ago

  • Target version changed from Development 2023-05-10 sprint to Development 2023-05-24 sprint
Actions #12

Updated by Peter Amstutz 11 months ago

(attached a version of lots-of-inputs that you can load with arv container_request create -o lots-of-inputs.json)

Stephen Smith wrote in #note-9:

Changes at arvados-workbench2|3e18e35172808479d02e02985563d658fec7bb0f branch 20424-io-panel-performance

  • Memoized the process io preview component, reduces re-renders from ~140 to 6
  • Rewrote file array processing code to use a single loop, cuts processing time in half (~5s to ~2s)
  • The remaining performance impact here seems to be solely due to having to run checks on all 4000 entries, each execution is <1ms

The remaining performance issues seem to stem mostly from the rendering code on process-io-card.tsx L400 and having to render 4000 entries. I'm not sure how to further optimize that part

I compared main with this branch and it is a huge improvement. In the main branch, the browser basically hangs for tens of seconds. It's unusable.

With this branch, the browser still stutters, but it's only for about 5 or 6 seconds. So while it's not ideal, it's on the outer end of tolerable. Lucas is correct that using a virtualized list to limit rendering to just the visible portion would be even more performant, but it would take more time to implement, and we have other bugs and performance issues to deal with.

This LGTM.

Actions #13

Updated by Stephen Smith 11 months ago

  • Status changed from In Progress to Resolved
Actions #14

Updated by Peter Amstutz 11 months ago

  • Release set to 63
Actions #15

Updated by Peter Amstutz about 1 month ago

  • Related to Bug #21525: Process view unusable on running containers that are too verbose added
Actions #16

Updated by Peter Amstutz about 1 month ago

  • Related to deleted (Bug #21525: Process view unusable on running containers that are too verbose)
Actions #17

Updated by Peter Amstutz about 1 month ago

  • Related to Bug #21508: Browser struggles with very large number of input or output parameters added
Actions

Also available in: Atom PDF