Arvados

We now have to pipelines replacing the previous one:

• https://ci.arvados.org/job/developer-run-tests-services-workbench2-components/
• https://ci.arvados.org/job/developer-run-tests-services-workbench2-e2e/

With this change we save between 8 to 10 minutes on every run (around 21% of the total time).

Here's a successful run: developer-run-tests: #4559

Lucas Di Pentima wrote in #note-3:

With this change we save between 8 to 10 minutes on every run (around 21% of the total time).

Which is still about ten minutes slower than it was before the big Workbench test change. :/ But this LGTM, thanks.

Actually sorry, looking at the other ticket I do have a couple ideas, but they're just ideas, you definitely did the ticket.

I think it would be nice if the Jenkins jobs were named …-services-workbench2-units and …-services-workbench2-integration, to better match the names used in run-tests.sh and provide more consistency for us.

We actually could ditch the units/components job entirely, and instead let it run from run-tests-remainder by changing its --skip arguments. The combination should still run faster than the integration tests, so it wouldn't increase total turnaround time, and this would let us keep more worker slots free for other work. This isn't critical though, it's a question of how we want to balance more human-friendly organization vs. more optimized configuration, and I don't need to have the final say on that.

Brett Smith wrote in #note-5:

I think it would be nice if the Jenkins jobs were named …-services-workbench2-units and …-services-workbench2-integration, to better match the names used in run-tests.sh and provide more consistency for us.

Good idea.

We actually could ditch the units/components job entirely, and instead let it run from run-tests-remainder by changing its --skip arguments. The combination should still run faster than the integration tests, so it wouldn't increase total turnaround time, and this would let us keep more worker slots free for other work. This isn't critical though, it's a question of how we want to balance more human-friendly organization vs. more optimized configuration, and I don't need to have the final say on that.

Related to your first comment, the time savings are not that great, and now I'm wondering if the newer Cypress is slower than before.

• developer-run-tests-services-workbench2-e2e: #4 - ~6 minutes
• developer-run-tests-services-workbench2-components: #6 - ~5 minutes

So definitely we're losing no trivial amount of time by splitting them. Setting the units/component job to run from the remainder job is a good idea, probably will just add ~10 minutes to that subjob and as you say, keep a free slot for other runs.

developer-run-tests: #4560

• Removed the workbench2_components job from developer-run-tests
• Set the developer-run-tests-remainder job to run the above

• Renamed developer-run-tests-services-workbench2-e2e to developer-run-tests-services-workbench2-integration
• Removed developer-run-tests-services-workbench2-components

These developer-run-tests-* should be renamed to just run-tests-*, but we still have the old run-tests-remainder job existing. If there's no value of keeping those old runs, I think we can remove it (or maybe rename it and remove it 30 days later?) so that the developer-run-test-remainder job can take its place.

Renamed the old run-tests-remainder to run-tests-remainder-old and the 3 developer-run-tests-* to run-tests-*.