Arvados

Similar to workbench_units, workbench_functionals, workbench_integration, it would be helpful if we further split workbench_integration tests into multiple jobs. Currently, the workbench_integration is the longest job in jenkins, taking around 21 mins. Breaking it into multiple jobs that can run parallel will help reduce the overall jenkins run time.

A couple options:

1. minitest 5.10.1 has --exclude option. This option can be used to exclude tests with a specific pattern such "test_project..." from a test run. Unfortunately, the pattern matching seems to be available only on the test name, not a test file name. To use this option we would need to edit our test files and make sure all test names start with a specific string. For example, to omit all tests from pipeline_instances_test.rb from a job run, we would need to update all tests in this file to have a name such as "test 'pipeline blah blah blah'"

2. Another option is to rename all the integration test files to something like 1_pipeline..., 1_projects...., 2_anonymous... We can then run all files starting with "1_" as one job and all files starting with "2_" as a separate job.

Branch 11251-split-wb-integration-tests takes an initial stab at using minutest --exclude option.

This branch intends to split the workbench integration tests into two jobs: one job running all tests in the test/integration/p*rb and another test running all other tests from all other files. (However, the tests in the p*.rb files themselves are not yet renamed to use specific prefixes)

IMO we should be fixing slow/redundant test cases before we give up and make the test pipeline/server fatter. It looks like the 10 slowest tests account for 3 minutes of runtime, and 6 of them test the sharing feature. It seems very likely something wasting a lot of time in either the tests or the sharing feature itself.

curl 'https://ci.curoverse.com/job/run-tests-apps-workbench-integration/699/consoleText' | perl -ne 'print "$1 $_" if /(\S+) s =/' | sort -n

...
7.01 WebsocketTest#test_test_running_job_with_too_many_previous_log_records = 7.01 s = .
7.05 SearchBoxTest#test_test_search_box_for_user_admin = 7.05 s = .
7.17 RepositoriesTest#test_browse_repository_from_jobs#show = 7.17 s = .
7.80 WebsocketTest#test_dashboard_arv-refresh-on-log-event = 7.80 s = .
8.41 ProjectsTest#test_scroll_collections_tab_for_project_with_201_collections_with_201_objects_with_ascending_sort_(case_insensitive) = 8.41 s = .
9.14 ProjectsTest#test_select_all_and_unselect_all_actions = 9.14 s = .
9.30 ProjectsTest#test_selection_Move_->_nil_for_project = 9.30 s = .
9.37 SearchBoxTest#test_test_search_box_for_user_active = 9.37 s = .
9.38 CollectionUploadTest#test_Upload_two_empty_files_with_the_same_name = 9.38 s = .
9.43 PipelineInstancesTest#test_Run_pipeline_instance_in_true_with_Two_Part_Pipeline_Template_with_collection_with_no_name_in_aproject_file_false = 9.43 s = .
9.48 PipelineInstancesTest#test_Run_pipeline_instance_in_true_with_Two_Part_Pipeline_Template_with_foo_collection_in_aproject_file_false = 9.48 s = .
9.52 UsersTest#test_unsetup_active_user = 9.52 s = .
9.66 PipelineInstancesTest#test_Run_a_pipeline_from_dashboard = 9.66 s = .
9.81 SmokeTest#test_all_first-level_links_succeed = 9.81 s = .
10.32 ProjectsTest#test_selection_Copy_->_nil_for_project = 10.32 s = .
10.37 PipelineInstancesTest#test_Run_pipeline_instance_in_false_with_Two_Part_Pipeline_Template_with_foo_collection_in_aproject_file_false = 10.37 s = .
10.37 UsersTest#test_setup_the_active_user = 10.37 s = .
10.57 ProjectsTest#test_test_search_all_projects_menu_item_in_projects_menu = 10.57 s = .
10.89 PipelineInstancesTest#test_Run_pipeline_instance_in_true_with_Two_Part_Template_with_dataclass_File_with_foo_collection_in_aproject_file_true = 10.89 s = .
10.93 LoginsTest#test_login_with_api_token_works_after_redirect = 10.93 s = .
11.13 ApplicationLayoutTest#test_getting_started_help_menu_item_false = 11.13 s = .
11.54 PipelineInstancesTest#test_Run_pipeline_instance_in_false_with_Two_Part_Template_with_dataclass_File_with_foo_collection_in_aproject_file_true = 11.54 s = .
12.05 ApplicationLayoutTest#test_getting_started_help_menu_item_true = 12.05 s = .
13.42 RepositoriesTest#test_active_can_manage_sharing_for_another_user = 13.42 s = .
13.65 RepositoriesTest#test_active_can_manage_sharing_for_another_group = 13.65 s = .
13.83 PipelineInstancesTest#test_Create_pipeline_inside_a_project_and_run = 13.83 s = .
14.43 ProjectsTest#test_copy_action_is_disabled_when_a_subproject_is_selected = 14.43 s = .
15.21 ProjectsTest#test_first_tab_loads_data_when_visiting_other_tab_directly = 15.21 s = .
16.66 ProjectsTest#test_Create_a_project_and_move_it_into_a_different_project = 16.66 s = .
17.71 PipelineInstancesTest#test_Create_and_run_a_pipeline = 17.71 s = .
17.79 RepositoriesTest#test_admin_can_manage_sharing_for_another_group = 17.79 s = .
17.91 RepositoriesTest#test_admin_can_manage_sharing_for_another_user = 17.91 s = .
19.90 ProjectsTest#test_project_owner_can_manage_sharing_for_another_user = 19.90 s = .
20.01 ProjectsTest#test_project_owner_can_manage_sharing_for_another_group = 20.01 s = .