Story #10077

[Workbench] reduce time required to run integration tests

Added by Tom Morris over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
09/29/2016
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-

Subtasks

Task #10166: Review 10077-workbench-integration-testsResolvedLucas Di Pentima

Associated revisions

Revision a894a61c
Added by Tom Morris over 2 years ago

Merge branch '10077-workbench-integration-tests' closes #10077

History

#1 Updated by Tom Morris over 2 years ago

Jenkins job run-tests-apps-workbench-integration is currently the longest leg of the test suite at 28 minutes. Every minute saved here is a minute shaved off developer iteration time.
./build/run-tests.sh --only apps/workbench_integration

Here are some timings from a recent run: https://ci.curoverse.com/job/run-tests-apps-workbench-integration/126/

  • Six tests of a minute apiece (fixing one would likely fix them all):

ProjectsTest#test_project_owner_can_manage_sharing_for_another_group = 49.47 s = . 49.47
ProjectsTest#test_project_owner_can_manage_sharing_for_another_user = 49.44 s = . 49.44
RepositoriesTest#test_active_can_manage_sharing_for_another_group = 53.31 s = . 53.31
RepositoriesTest#test_active_can_manage_sharing_for_another_user = 52.84 s = . 52.84
RepositoriesTest#test_admin_can_manage_sharing_for_another_group = 57.57 s = . 57.57
RepositoriesTest#test_admin_can_manage_sharing_for_another_user = 57.81 s = . 57.81

  • Next longest test at 32 seconds (but perhaps slow because create pipeline instance is slow):

PipelineInstancesTest#test_Create_and_run_a_pipeline = 32.04 s = . 32.04

  • Ten tests in the 15-30 second range include a bunch of pipeline stuff:

UserSettingsMenuTest#test_create_new_repository = 16.68 s = . 16.68
UsersTest#test_create_a_new_user = 17.65 s = . 17.65
PipelineInstancesTest#test_Create_pipeline_inside_a_project_and_run = 26.28 s = . 26.28
PipelineInstancesTest#test_Run_a_pipeline_from_dashboard = 21.57 s = . 21.57
PipelineInstancesTest#test_Run_pipeline_instance_in_false_with_Two_Part_Pipeline_Template_with_foo_collection_in_aproject_file_false = 23.40 s = . 23.4
PipelineInstancesTest#test_Run_pipeline_instance_in_false_with_Two_Part_Template_with_dataclass_File_with_foo_collection_in_aproject_file_true = 23.15 s = . 23.15
PipelineInstancesTest#test_Run_pipeline_instance_in_true_with_Two_Part_Pipeline_Template_with_collection_with_no_name_in_aproject_file_false = 21.08 s = . 21.08
PipelineInstancesTest#test_Run_pipeline_instance_in_true_with_Two_Part_Pipeline_Template_with_foo_collection_in_aproject_file_false = 19.89 s = . 19.89
PipelineInstancesTest#test_Run_pipeline_instance_in_true_with_Two_Part_Template_with_dataclass_File_with_foo_collection_in_aproject_file_true = 20.86 s = . 20.86
ProjectsTest#test_copy_action_is_disabled_when_a_subproject_is_selected = 15.25 s = . 15.25

  • 15 tests in the 10-15 sec. range:

UserSettingsMenuTest#test_test_user_settings_menu_for_user_active = 10.46 s = . 10.46
UsersTest#test_setup_the_active_user = 10.82 s = . 10.82
UsersTest#test_unsetup_active_user = 10.19 s = . 10.19
PipelineInstancesTest#test_enter_a_float_for_a_number_pipeline_input = 11.42 s = . 11.42
ProjectsTest#test_Create_a_project_and_move_it_into_a_different_project = 12.39 s = . 12.39
ProjectsTest#test_selection_Move_->_nil_for_project = 11.43 s = . 11.43
ProjectsTest#test_test_search_all_projects_menu_item_in_projects_menu = 12.18 s = . 12.18
DownloadTest#test_download_anonymous_content_from_keep-web_by_portable_data_hash = 10.73 s = . 10.73
DownloadTest#test_download_from_keep-web_using_a_session_token = 12.97 s = . 12.97
CollectionUploadTest#test_Upload_non-empty_files = 11.69 s = . 11.69
CollectionUploadTest#test_Upload_two_empty_files_with_the_same_name = 11.29 s = . 11.29
WebsocketTest#test_pipeline_instance_arv-refresh-on-log-event = 11.55 s = . 11.55
WebsocketTest#test_test_live_logging_and_scrolling_for_pipeline_instances = 10.06 s = . 10.06
ApplicationLayoutTest#test_getting_started_help_menu_item_false = 11.20 s = . 11.2
ApplicationLayoutTest#test_getting_started_help_menu_item_true = 11.86 s = . 11.86

#2 Updated by Tom Morris over 2 years ago

  • Target version set to 2016-10-12 sprint

#3 Updated by Tom Morris over 2 years ago

  • Assigned To set to Tom Morris

#4 Updated by Tom Morris over 2 years ago

  • Status changed from New to In Progress

#5 Updated by Lucas Di Pentima over 2 years ago

This looks good to me.
One small observation: Could you explain on the code what's the side effect of removing Capybara.default_max_wait_time so we can later investigate based on that finding?

#6 Updated by Tom Morris over 2 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:a894a61cc9004e184bc63f9f0a785f555d6c6829.

Also available in: Atom PDF