Idea #6053: [Workbench] [Performance] Dashboard display is sluggish - Arvados

Actions

Copy link

Idea #6053

closed

[Workbench] [Performance] Dashboard display is sluggish

Added by Radhika Chippada almost 9 years ago. Updated about 7 years ago.

Status:

Resolved

Priority:

Normal

Assigned To:

Category:

Performance

Target version:

Start date:

05/18/2015

Due date:

Story points:

Description

Dashboard display takes several seconds to display. When looked at the workbench console log (pointing to qr1hi API server), the following observations were made (which can be potential areas of improvement):

It seems like we are making several calls to API server for one collection and one link at a time

API client: 0.000334074 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-iz4x9s864e411js"]}  
API client: 0.066898288 API transaction
API client: 0.000214607 Parse response
API client: 0.000395874 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-iz4x9s864e411js"]]] 
API client: 0.064143092 API transaction
API client: 0.000176 Parse response
API client: 0.000399506 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-79rrud51ezmvck4"]}  
API client: 0.070629777 API transaction
API client: 0.000173597 Parse response
API client: 0.000306681 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-79rrud51ezmvck4"]]] 
...

It appears that several calls are made to /nodes?

API client: 0.000246677 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/nodes    
API client: 0.253671224 API transaction
API client: 0.003016469 Parse response
API client: 0.000431834 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/nodes    
API client: 0.205285103 API transaction
API client: 0.001184667 Parse response
API client: 0.000465309 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/nodes    
...

Complete workbench log

Started GET "/" for 127.0.0.1 at 2015-05-17 20:50:31 -0400
Processing by ProjectsController#index as HTML
API client: 0.000878306 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/users/current    
API client: 0.232379553 API transaction
API client: 0.000285227 Parse response
API client: 0.000282894 Prepare request https://qr1hi.arvadosapi.com/discovery/v1/apis/arvados/v1/rest    
API client: 0.835217807 API transaction
API client: 0.006798759 Parse response
API client: 0.000270661 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/pipeline_instances   [["state","in",["RunningOnServer","RunningOnClient"]]] ["started_at asc","created_at asc"]
API client: 0.078352693 API transaction
API client: 0.000133876 Parse response
API client: 0.000249674 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/pipeline_instances   [["state","in",["RunningOnServer","RunningOnClient"]]] ["started_at asc","created_at asc"]
API client: 0.073826468 API transaction
API client: 0.000130813 Parse response
API client: 0.000266825 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/pipeline_instances   [["state","in",["Complete","Failed","Paused"]],["finished_at","!=",null]] ["finished_at desc"]
API client: 0.192317137 API transaction
API client: 0.001960185 Parse response
API client: 0.000343024 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/pipeline_templates  {"uuid":["qr1hi-p5p6p-ftcb0o61u4yd2zr","qr1hi-p5p6p-ftcb0o61u4yd2zr","qr1hi-p5p6p-ftcb0o61u4yd2zr","qr1hi-p5p6p-ftcb0o61u4yd2zr","qr1hi-p5p6p-ftcb0o61u4yd2zr","qr1hi-p5p6p-gjag73dz4eazg1u","qr1hi-p5p6p-gjag73dz4eazg1u","qr1hi-p5p6p-gjag73dz4eazg1u"]}  
API client: 0.089368034 API transaction
API client: 0.000381734 Parse response
API client: 0.000435583 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/pipeline_instances   [["state","in",["RunningOnServer","RunningOnClient"]]] ["started_at asc","created_at asc"]
API client: 0.067580539 API transaction
API client: 0.000151119 Parse response
API client: 0.00029224 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/pipeline_instances   [["state","in",["Complete","Failed","Paused"]],["finished_at","!=",null]] ["finished_at desc"]
API client: 0.170014311 API transaction
API client: 0.002306176 Parse response
  Rendered application/_pipeline_status_label.html.erb (0.8ms)
API client: 0.000305501 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-hqfxewp0r3dsaov"]}  
API client: 0.069504503 API transaction
API client: 0.000273489 Parse response
API client: 0.000241297 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-hqfxewp0r3dsaov"]]] 
API client: 0.062257167 API transaction
API client: 0.000137466 Parse response
  Rendered application/_pipeline_status_label.html.erb (0.2ms)
  Rendered application/_pipeline_status_label.html.erb (0.2ms)
  Rendered application/_pipeline_status_label.html.erb (0.1ms)
  Rendered application/_pipeline_status_label.html.erb (0.1ms)
  Rendered application/_pipeline_status_label.html.erb (0.1ms)
API client: 0.000334074 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-iz4x9s864e411js"]}  
API client: 0.066898288 API transaction
API client: 0.000214607 Parse response
API client: 0.000395874 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-iz4x9s864e411js"]]] 
API client: 0.064143092 API transaction
API client: 0.000176 Parse response
API client: 0.000399506 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-79rrud51ezmvck4"]}  
API client: 0.070629777 API transaction
API client: 0.000173597 Parse response
API client: 0.000306681 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-79rrud51ezmvck4"]]] 
API client: 0.066004533 API transaction
API client: 0.000132684 Parse response
API client: 0.000357527 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-p2t9jvnpggur7vw"]}  
API client: 0.075255235 API transaction
API client: 0.000223686 Parse response
API client: 0.000333217 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-p2t9jvnpggur7vw"]]] 
API client: 0.067465995 API transaction
API client: 0.000129648 Parse response
API client: 0.000284067 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-2t01k1kema7epyj"]}  
API client: 0.072013175 API transaction
API client: 0.000196958 Parse response
API client: 0.000353564 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-2t01k1kema7epyj"]]] 
API client: 0.063153464 API transaction
API client: 0.000159383 Parse response
API client: 0.000457349 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-wembfqx39kfyci5"]}  
API client: 0.072191773 API transaction
API client: 0.00018893 Parse response
API client: 0.000382225 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-wembfqx39kfyci5"]]] 
API client: 0.05986727 API transaction
API client: 0.000211157 Parse response
API client: 0.000314946 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-7cerpxo1u39gncx"]}  
API client: 0.067371118 API transaction
API client: 0.000259134 Parse response
API client: 0.000355187 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-7cerpxo1u39gncx"]]] 
API client: 0.063150057 API transaction
API client: 0.000191293 Parse response
API client: 0.000293813 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-nzraz1de84r1t07"]}  
API client: 0.071077866 API transaction
API client: 0.000487389 Parse response
API client: 0.000291013 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-nzraz1de84r1t07"]]] 
API client: 0.063935105 API transaction
API client: 0.000208452 Parse response
API client: 0.000326847 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-iqg5cl8gfku8rzx"]}  
API client: 0.068502243 API transaction
API client: 0.00018937 Parse response
API client: 0.000607518 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-iqg5cl8gfku8rzx"]]] 
API client: 0.060600105 API transaction
API client: 0.000171186 Parse response
API client: 0.000296161 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-7r7p4a34zgjz87p"]}  
API client: 0.066579532 API transaction
API client: 0.000207053 Parse response
API client: 0.000430108 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-7r7p4a34zgjz87p"]]] 
API client: 0.059098428 API transaction
API client: 0.000144929 Parse response
  Rendered application/_pipeline_status_label.html.erb (0.3ms)
API client: 0.000719445 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-r5hm291d78gf8yd"]}  
API client: 0.068647599 API transaction
API client: 0.000478276 Parse response
API client: 0.000307643 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-r5hm291d78gf8yd"]]] 
API client: 0.062027912 API transaction
API client: 0.000160487 Parse response
API client: 0.000306887 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-yeiw06g44a04tdg"]}  
API client: 0.092978051 API transaction
API client: 0.000213342 Parse response
API client: 0.000401655 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-yeiw06g44a04tdg"]]] 
API client: 0.072429915 API transaction
API client: 0.000266741 Parse response
API client: 0.000465145 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-47cunhdr9mxx1dr"]}  
API client: 0.077359384 API transaction
API client: 0.000375746 Parse response
API client: 0.000124052 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-47cunhdr9mxx1dr"]]] 
API client: 0.067963481 API transaction
API client: 0.000141627 Parse response
API client: 0.000315552 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-3ge2ejz4iyce4pp"]}  
API client: 0.068210023 API transaction
API client: 0.000268424 Parse response
API client: 0.000314701 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-3ge2ejz4iyce4pp"]]] 
API client: 0.063047793 API transaction
API client: 0.000164833 Parse response
API client: 0.000284666 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-v6b5zycmdtirjrw"]}  
API client: 0.066684741 API transaction
API client: 0.000222602 Parse response
API client: 0.000355972 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-v6b5zycmdtirjrw"]]] 
API client: 0.059433537 API transaction
API client: 0.000162993 Parse response
API client: 0.000474117 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-8kx603ceg2xih5s"]}  
API client: 0.067639708 API transaction
API client: 0.000257302 Parse response
API client: 0.000367928 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-8kx603ceg2xih5s"]]] 
API client: 0.062623667 API transaction
API client: 0.000135324 Parse response
API client: 0.00044936 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-i7bt0y50t4wolfy"]}  
API client: 0.071481153 API transaction
API client: 0.000172758 Parse response
API client: 0.000455209 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-i7bt0y50t4wolfy"]]] 
API client: 0.063030312 API transaction
API client: 0.000217415 Parse response
  Rendered application/_pipeline_status_label.html.erb (0.2ms)
API client: 0.000326496 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-udnme0d7okcgrw3"]}  
API client: 0.071815087 API transaction
API client: 0.000204767 Parse response
API client: 0.00050553 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-udnme0d7okcgrw3"]]] 
API client: 0.062109233 API transaction
API client: 0.00015839 Parse response
API client: 0.000277025 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-m5zf91uq47mfhc4"]}  
API client: 0.067411208 API transaction
API client: 0.000184179 Parse response
API client: 0.000339182 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-m5zf91uq47mfhc4"]]] 
API client: 0.088670602 API transaction
API client: 0.000156615 Parse response
API client: 0.000288672 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-dtmkc7n6017br8z"]}  
API client: 0.066239907 API transaction
API client: 0.00106025 Parse response
API client: 0.001208536 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-dtmkc7n6017br8z"]]] 
API client: 0.061351453 API transaction
API client: 0.000159923 Parse response
API client: 0.000380737 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-fk3dyi2h3yfrvd3"]}  
API client: 0.073061728 API transaction
API client: 0.000162584 Parse response
API client: 0.00035452 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-fk3dyi2h3yfrvd3"]]] 
API client: 0.066352994 API transaction
API client: 0.000150181 Parse response
API client: 0.00019883 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"uuid":["qr1hi-4zz18-qesjzq1ckbaftbl"]}  
API client: 0.083504786 API transaction
API client: 0.000164868 Parse response
API client: 0.000339534 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/links   [["head_uuid","in",["qr1hi-4zz18-qesjzq1ckbaftbl"]]] 
API client: 0.063078433 API transaction
API client: 0.000140794 Parse response
API client: 0.000117054 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/jobs/queue_size    
API client: 0.070813775 API transaction
API client: 0.021508741 request_time
API client: 0.000218616 Parse response
API client: 0.000246677 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/nodes    
API client: 0.253671224 API transaction
API client: 0.003016469 Parse response
API client: 0.000431834 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/nodes    
API client: 0.205285103 API transaction
API client: 0.001184667 Parse response
API client: 0.000465309 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/nodes    
API client: 0.314293377 API transaction
API client: 0.001789129 Parse response
API client: 0.000421827 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/nodes    
API client: 0.249869629 API transaction
API client: 0.001396601 Parse response
  Rendered projects/_compute_node_summary.html.erb (1549.0ms)
API client: 0.000140962 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/jobs/queue    
API client: 0.088902398 API transaction
API client: 0.000139759 Parse response
API client: 0.000400243 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/nodes    
API client: 0.260784223 API transaction
API client: 0.002231811 Parse response
API client: 0.000572629 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/nodes    
API client: 0.212883949 API transaction
API client: 0.001281634 Parse response
  Rendered projects/_compute_node_status.html.erb (800.4ms)
API client: 0.000211429 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections   [["owner_uuid","is_a","arvados#group"]] ["modified_at desc"]
API client: 0.127554924 API transaction
API client: 0.000305983 Parse response
API client: 0.000586701 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/groups   [["uuid","in",["qr1hi-j7d0g-7qgmt2zx4g29jit","qr1hi-j7d0g-7qgmt2zx4g29jit","qr1hi-j7d0g-7qgmt2zx4g29jit","qr1hi-j7d0g-7qgmt2zx4g29jit","qr1hi-j7d0g-7qgmt2zx4g29jit","qr1hi-j7d0g-7qgmt2zx4g29jit","qr1hi-j7d0g-7qgmt2zx4g29jit","qr1hi-j7d0g-7qgmt2zx4g29jit"]]] 
API client: 0.066978584 API transaction
API client: 0.000172905 Parse response
API client: 0.000217862 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections   [["owner_uuid","is_a","arvados#group"]] ["modified_at desc"]
API client: 0.10718079 API transaction
API client: 0.000292406 Parse response
  Rendered projects/_show_dashboard.html.erb (6584.8ms)
  Rendered projects/index.html.erb within layouts/application (6587.5ms)
API client: 0.000336571 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/authorized_keys  {"authorized_user_uuid":"qr1hi-tpzed-ktpvhqu89qoib9f"}  
API client: 0.068924031 API transaction
API client: 0.000283224 Parse response
API client: 0.000204225 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/collections  {"created_by":"qr1hi-tpzed-ktpvhqu89qoib9f"}  
API client: 0.125075929 API transaction
API client: 0.000211763 Parse response
API client: 0.000376168 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/pipeline_instances  {"created_by":"qr1hi-tpzed-ktpvhqu89qoib9f"}  
API client: 0.079374417 API transaction
API client: 0.000469808 Parse response
API client: 0.000372106 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/groups   [["group_class","=","project"]] name
API client: 0.80578422 API transaction
API client: 0.002824583 Parse response
API client: 0.000435404 Prepare request https://qr1hi.arvadosapi.com/arvados/v1/groups   [["group_class","=","project"]] name
API client: 0.733503236 API transaction
API client: 0.018161996 Parse response
  Rendered application/_projects_tree_menu.html.erb (1988.8ms)
  Rendered application/_browser_unsupported.html (0.5ms)
  Rendered getting_started/_getting_started_popup.html.erb (3.4ms)
  Rendered layouts/body.html.erb (2292.4ms)
Completed 200 OK in 10106ms (Views: 8991.8ms | ActiveRecord: 0.0ms)

Started GET "/assets/application.js" for 127.0.0.1 at 2015-05-17 20:50:41 -0400
Started GET "/assets/application.css" for 127.0.0.1 at 2015-05-17 20:50:41 -0400

Started GET "/assets/pipeline-running.gif" for 127.0.0.1 at 2015-05-17 20:50:41 -0400
Started GET "/assets/mouse-move.gif" for 127.0.0.1 at 2015-05-17 20:50:41 -0400

Files

perf-profile-dashboard.png (433 KB) perf-profile-dashboard.png

Radhika Chippada, 05/18/2015 01:02 AM

Actions

Copy link

Updated by Radhika Chippada about 7 years ago

Status changed from New to Resolved

Now the collections and links are retrieved in one call. And the redundant nodes calls are eliminated.

Actions

Copy link

Also available in: Atom PDF

Project

General

Profile

Arvados

Custom queries

Idea #6053

[Workbench] [Performance] Dashboard display is sluggish

Updated by Radhika Chippada about 7 years ago