Bug #17500

Fix flaky integration tests

Added by Peter Amstutz 27 days ago. Updated about 13 hours ago.

Status:
In Progress
Priority:
Normal
Assigned To:
Category:
Workbench2
Start date:
04/20/2021
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Story points:
-

Description

The following tests tend to fail with "Timed out retrying after 4000ms"

favorites.spec.js
1) can copy selected into the collection

This one too:

sharing.spec.js
1) can share projects to other users

Also

page-not-found.spec.js 
1) Page not found tests

Subtasks

Task #17544: Review 17500-cypress-flakyness-fixesIn ProgressPeter Amstutz

History

#1 Updated by Peter Amstutz 27 days ago

  • Description updated (diff)

#2 Updated by Ward Vandewege 22 days ago

  • Description updated (diff)

#3 Updated by Peter Amstutz 21 days ago

  • Assigned To set to Ward Vandewege

#4 Updated by Lucas Di Pentima 13 days ago

I've been looking for methods to reduce test flakyness. I think that one of the causes is the fact that we use cy.visit() to go to specific URLs. This makes the browser go to the url and reloading the entire app in the process... which introduces delays because on app boostrap, there're lots of things going on in the background.

One idea would be to stop using cy.visit() and somehow access the app's router from Cypress, so we can push new URLs to the browser's history and let the app react to it, this I think will improve response times a lot. I've found a couple of hints on how to do this:

https://github.com/cypress-io/cypress/issues/128#issuecomment-406155816

and more react specific:

https://github.com/cypress-io/cypress/issues/3120#issuecomment-453254393

#5 Updated by Peter Amstutz 8 days ago

  • Target version changed from 2021-04-14 sprint to 2021-04-28 bughunt sprint

#6 Updated by Peter Amstutz 7 days ago

  • Description updated (diff)

#7 Updated by Peter Amstutz 7 days ago

  • Assigned To changed from Ward Vandewege to Lucas Di Pentima

#8 Updated by Lucas Di Pentima 6 days ago

  • Status changed from New to In Progress

#9 Updated by Lucas Di Pentima about 13 hours ago

Updates at arvados-workbench2|e76af0d - branch 17500-cypress-flakyness-fixes

Test runs: Fixes:
  • Avoids using multi-slash URLs on Cypress requests.
  • Fixes issue that made the 'page not found' message to not be displayed on unknown routes.
  • Adds app router navigation capabilities to Cypress tests, to improve reliability and speed.
  • Adds object cleanup before every test to avoid potential issues and test slowness.
  • Fixes cy.loginAs() command to avoid leaking session data if used more than once on a single test.
  • Fixes favorites suite test that relied on the above bug to sometimes pass.

Also available in: Atom PDF